Evolution of Business Intelligence

What is business intelligence (BI)?

Companies use Business Intelligence services to achieve their digital transformation objectives. These tools are mostly cloud-based and composed of different applications, such as reporting tools, statistical analysis tools, database management systems, and data mining applications. By using these technologies, businesses can make informed decisions based on the insights they gain from their data. With a BI platform, organizations can store, classify, and search through their datasets quickly and easily - allowing them to access specific insights that will help them reach their business goals.

‍BI technology has the potential to leverage a company's performance. Beyond just analytics, this platform can allow collaboration between all members of the organization, both internally and externally. This means that stakeholders such as customers, partners, suppliers and other external actors can also get on board with a company's internal processes.

The early days: Information is BI

The earliest published work that used the term BI, mentions how banker Sir Henry Furnese profited by maintaining “a perfect train of business intelligence” that informed him about the outcome of war. Hence, the earliest form of BI was simply valuable information that was unavailable to competitors.

If businesses did collect data, they were stored on paper. This data could be analysed in a rudimentary fashion to gain insights. However, this process was expensive and labour intensive. Leveraging available data was therefore restricted to big businesses.

IT enters the picture: Reporting with data

Starting in the pre-digital age of the late 1950s, the availability of computers changed the way businesses approached BI. It was no longer prohibitively expensive to analyse accumulated business data. IT teams were the centre of this early stage evolution of business intelligence systems. An elite team of mathematicians, engineers and computer specialists – George Dantzig, Douglas Engelbart and Jay Forrester pioneered the decision support systems (DSS) of the era. DSS was the precursor of BI as we know it now.

The earliest known use of a computerized DSS is in 1951. Lyons Tea Shops in UK used the LEO I (Lyons Electronic Office I) digital computer. LEO I software factored in the weather forecast to help determine the goods carried by “fresh produce” delivery vans to Lyons’s UK shops

The DSSs slowly evolved into the larger and more complex Management Information Systems (MIS) enabled by the processing power of IBM 360. MIS focused on providing managers with structured, periodic reports derived from accounting and transaction systems. In addition, DSS themselves split up in 5 types, each targeted at different user groups within an organization. 30 years after LEO I, researchers considered decision support systems a new class of information systems.

Advancements and Evolution into the late 1980s

Beginning with the first use of computers in business, companies had an opportunity to store data in a new way. However, with the invention of hard disks by IBM in 1956, mass data storage became possible and genuinely changed how large businesses kept track of their documents and records.

From this point on, different storage technologies such as floppy discs and laser discs meant that not only could even more significant amounts of data be stored, but software devices were also created for managing this data through decision support systems (DSS). This prompted a range of vendors during the 70s to develop tools to access and make relevant deductions from this large data pool.

Despite these breakthroughs, BI analysis still needed to be simplified due to its many limitations. A 1988 international conference was held for streamlining these processes called the Multiway Data Analysis consortium in Rome, which served as a significant milestone towards making efficient BI analysis more user-friendly.

Changing Times In The 1980s And 1990s

In 1989, Howard Dresner of Gartner popularized “business intelligence services” as a solution to data storage and analysis needs such as DSS and EIS. This led to a revolution in the field, with mergers and acquisitions causing more advanced tools, such as data warehouses, to be introduced. The time it took for data retrieval was vastly cut down due to data warehousing, which condensed all stored information into a single location. This marked the beginning of the modern phase of business intelligence services.

The introduction of data warehousing marked the beginning of business intelligence (BI) as we know it today. It also brought about core components of BI like Extract, Transform and Load (ETL) tools and Online Analytical Processing (OLAP). This phase of development was referred to as Business Intelligence 1.0.

Arrival of the World Wide Web

As the digital era dawned, the world wide web went mainstream, handheld computing technology progressed by leaps and bounds and “customers” of BI changed. Organizations wanted faster access to reports. The access to BI was also expanding to non-technical users. The existing tools were extremely difficult to use as they were built for experts. In addition, data was stored in silos. In this scenario, going through IT for every single request was no longer feasible. The earliest BI tools offered to combat these two issues – complexity & time needed to obtain BI.

This was when data warehousing arrived on the scene. Consolidating the existing disparate data sources into one single structured location drastically cut the time it took to access data. This allowed for deeper dive and analysis of data to gain insights. This is the point where the IT centric system in BI began to diminish.

First wave disruption: Visual centric BI disrupts the market

Finally, all the parts of BI as we know it now were mainstream. This was when the first business intelligence wave of disruptors arrived. Software that simplified BI for end users took over the user stack. These products focused on data visualization. Instead of number-heavy, descriptive reports a business user could now consume the analysed data as dashboards that included interactive pie charts, bar charts etc.

Second wave disruption: Data Visualization to cloud computing

However, throughout this evolution in BI, the layer that remained virtually unchanged was the data layer. All BI tools used structured data from a data warehouse maintained by the business. This is no longer the case. The major deterrent for organizations to move to cloud computing was security. According to Forbes, the Manufacturing sector has the greatest uptake of Cloud Business Intelligence solutions with 58%, followed by Financial Services (40%) and Business Services (40%).

Big data and cloud computing has changed the ways in which data is structured and stored. Enterprise software vendors have increasingly viewed Cloud BI as a major component of their go-to-market strategies, with the percentage rising from 65% in 2019 to 72% after the pandemic. This trend is expected to continue and intensify, with up to 95% or more organizations considering Cloud BI as essential for distributed organizations in the post-COVID-19 world.

Cloud first BI tools like IBM Cognos Analytics, SAP BusinessObjects BI, Birst BI, Amazon QuickSight, Domo etc. now use incredible processing power, artificial intelligence, machine learning and predictive analytics to analyse huge volumes of data. A BI dashboard will now, in real-time, show actionable insights, identify patterns, showcase trends, predict future outcomes and even offer remedial insights to correct course.

We are now in the middle of the second wave of disruption – the era of cloud first, self-service BI. Yet, the rumblings of a third wave of disruptive technologies are already being heard.

The early trends of third wave disruption

AI & Natural Language Processing (NLP) technology are currently used to offer innovations in self-service BI. “40 percent of digital transformation initiatives will be supported by cognitive/AI capabilities, providing timely critical insights for new operating and monetization models,” predicts the International Data Corporation.

Some key trends in BI are:

• Auto narratives – insights/outcomes delivered in natural language
• BI bots – users can talk to and receive insights from specialized bots
• Mobile analytics – on demand, 365*24/7 wherever you are analytics
• Collaborative BI – users collaborate on BI platforms to gain insight
• Data governance – ensuring data quality for unstructured data (master data management).

With self-service, cloud first and pay as you use business models, IT focus has shifted towards efficient data storage, massive data processing, instant data availability and building cognitive capabilities; while business needs are shifting from reports to data. Leading cloud providers like AWS are building the required business intelligence platforms and services to help organizations take the leap towards the third wave disruption.

‍