BI tools: Three Generations

Introduction

BI tools play an incredibly important role in democratizing or “operationalizing” data at scale. The mission of Business Intelligence platforms is unsurprisingly to enlighten.. business decisions. There are two ways to go about this:

1. Provide an outstanding technical platform allowing the data team to self-serve quality data to operational teams.
2. Organize complete emancipation of operational teams from the data team. That is, ensure that business teams (= teams that are NOT the data team) can generate data-based insights 100% by themselves.

These two approaches stem from the same problem: operational teams often lack trust in the data, which results in relentless slack pings on the data team. Everyone is annoyed, and nothing works. The good news is that we’re making huge progress.

Although operational teams have gained a good deal of autonomy, data teams still get involved in too many organizations. Removing operational teams’ dependence on the data team is key: it allows the data team to focus on higher-impact stuff, and it allows other teams to move faster with their day-to-day work. And just like that, everyone is happier.

So how exactly do we make that happen? A good first step is to look at the evolution of Business Intelligence in the past decades. We’ve identified three generations of Business Intelligence tools: traditional BI, self-service BI, and augmented analytics. The evolution of BI tools has been characterized by the gradual emancipation of operational teams/domain experts from data teams and IT.

‍

‍

Let’s press pause for a second, so we can get on the same page about what Business Intelligence is (and what it isn’t). BI is a term used for the method and technology used to analyze data. BI uses information about your business to provide you with insights that help you make better decisions, such as how to improve your product line or cut costs. You can use this data to help you make decisions about what products to sell, where to open a new store, or which marketing strategies are working.

‍

Three generations of Business Intelligence tools

‍

BI 1.0: Traditional BI

Traditional BI is a database-centric approach to analyzing data; it relies on static dashboards composed of visualizations. The dashboards are defined well in advance, based on common business questions. Answering new questions requires time and technical skills. Big tech companies such as Microsoft, IBM, Oracle and SAP were the leaders of this BI era. The traditional BI techniques are, for the most part, outdated and no longer useful for modern-day businesses.

Traditional BI approaches include:

Reporting and Dashboards: This approach provides an overview of your business data and how it relates to business operations. It also allows users to manipulate data, explore relationships, and create reports that can be shared with others.

Data Warehousing: This approach involves storing all of your company’s data in one place and making it accessible for analysis through a database management system (DBMS). It allows you to analyze large volumes of data at a time, as well as perform advanced reporting functions such as drill-down reports.

The first generation of BI tools is characterized by a complete dependence of operational teams on the data team. Data teams are a complete bottleneck for every question involving data. This is for a very simple reason: only the data team has the skills necessary to extract insights from data. The first BI tools were rather complex, code-heavy platforms which only technical profiles could adopt; tables were queried in SQL, a language known at the time by mostly technical profiles.

The typical workflow of someone needing a report would be the following: The business person asks the data team for a report. The data team provides a static report which becomes outdated in the following days, so they ask again and again until they get their hands on something new—or just give up entirely!

‍

BI 2.0: Self-service

The second generation of BI tools fixed most of these issues.

The first step was to replace static reports with dynamic ones: dashboards or scorecards that could be updated regularly by users themselves without having to rely on the data team anymore. User-friendly self-service analytics platforms were developed as part of second-generation BI tools, allowing non-technical users to access all their data sources and generate reports and dashboards at their discretion. These new solutions were much easier to use than their predecessors. They allowed users who weren’t familiar with SQL code to easily get insights from their data without waiting for weeks or months before they got back an answer from the data team.

BI tools from this generation largely eliminated the tech stack and focused on providing discovery and visualization capabilities to their users. This effort empowered business users to uncover business insights on their own.

Another problem with the first generation of BI tools was that they were difficult to use and difficult to learn. Users had to spend time learning how to use these tools and then spend even more time trying to figure out what data they needed to import or extract from their data sources in order to get useful information. The second generation addressed this issue by making the tools easier for users to understand and use.

Although there has been a lot of improvement in the space, second-generation BI tools still had the business teams relying heavily on data teams for data preparation and modeling. Unsurprisingly, this is what the third generation of Business Intelligence tools tried to solve.

‍

BI 3.0: Augmented analytics

The third generation of BI kept its promise to further reduce the dependence of operational teams on the data team, by further removing the complexity from BI processes. This has mainly been achieved through.. augmented analytics. Augmented Analytics is “the use of enabling technologies such as machine learning and AI to assist with data preparation, insight generation and insight explanation to augment how people explore and analyze data in analytics and BI platforms” (Gartner glossary.

What does augmented analytics mean for operational teams?

I was discussing augmented analytics with Anna from Whaly who likened it to the “IKEA effect”. I’ve found this analogy to be super evocative. The IKEA effect states that we place a disproportionately high value on things we contributed to creating. When building your own IKEA wardrobe, you usually tend to value it more because you’ve invested time and effort in the building. And because YOU built it, it’s a great wardrobe.

It’s the same with augmented analytics. Augmented analytics empowers operational teams to own the full data pipeline. Yes, you heard correctly. This new capability enables users to create a self-service pipeline extracting data from various sources, transforming it into the format required by the reporting tool, and loading it into a data warehouse or other destination. Automation drastically simplifies each step of the process, completely demystifying data preparation for less technical profiles.

Back to the IKEA effect: when business profiles build their own automated data pipelines for extracting and modeling data themselves, they tend to trust the data more. They have put more effort into building the reports and fetching the analytics. They have been involved at every step of the way. This cognitive bias leads them to trust the data way more than when it has been served by another team.

A lot of people view this as the only way of achieving data democratization at scale. Instead of the data team providing clean data and pre-made dashboards in a self-serve manner, we shift towards a paradigm where operational teams can own the full pipeline: data collection, data modeling, visualization, etc. When building a dashboard from scratch, from data collection to clear insights, how could you not trust it? You’ve done all the work upstream. There are no trust issues to be had, no questions to be asked, no people to ping, or any other waste of time. Of course, this could not be achieved without augmented analytics.

What does augmented analytics mean for technical teams?

Augmented Analytics also increases the capabilities of BI tools catering to technical profiles. It can be used to find insights that would not be found using traditional analytics. For example, if you have an email list with a million email addresses and want to know which recipients are most likely to open your emails and click on the links within them, augmented analytics will look at all the features of each subscriber’s profile (such as their age, gender, location) and then use machine learning techniques to find patterns across all 1 million subscribers. Advanced analytics features allow users to access sophisticated methods for analyzing data that would otherwise be beyond their capabilities as individual business analysts or data analysts. For example, many platforms include advanced statistical analysis tools such as regression modeling and time series forecasting. Some also include machine learning algorithms for predictive modeling, which helps professionals make better-informed decisions about future performance based on existing trends in historical data (e.g., likelihood of customer churn).

Conclusion

Business Intelligence tools have gone through three main stages of evolution: Traditional BI, Self-Service BI, and Augmented Analytics. Through this evolution, some companies have preferred to focus on removing complexity from the BI process, granting autonomy independence to business teams from data teams (e.g: Whaly). Other platforms have opted to cater to technical profiles, leveraging augmented analytics to push analytics capabilities further. (e.g: Mode). Regardless of which way BI tools have gone, the goal has been to allow business teams to access and understand data whenever they need it - to be able to confidently make the right business decisions. Regardless of how this is achieved, an in-depth trust by business departments in data and data processes is the only way towards trust in their dashboards and other business reporting devices.

More modern data stack benchmarks?

Find more benchmarks and analysis on the modern data stack here. We write about all the processes involved when leveraging data assets: from the modern data stack to data teams composition, to data governance. Our blog covers the technical and the less technical aspects of creating tangible value from data.

At Castor, we are building a data documentation tool for the Notion, Figma, and Slack generation. Or data-wise for the Fivetran, Looker, Snowflake, DBT aficionados. We designed our catalog to be easy to use, delightful and friendly.‍

Want to check it out? Learn more about CastorDoc with a free demo.

‍

Subscribe to the Newsletter