Business intelligence is often treated as a way to find answers. But before focusing on the answer, there is another question organisations should consider: are we asking the right question in the first place?
Dashboards, reports and analytics tools can all help uncover patterns in business data, but they can also reinforce assumptions if users approach the information with a conclusion already in mind. A chart can look convincing, a trend can appear obvious and a relationship between two measures can seem meaningful, without necessarily demonstrating the correct conclusion.
This is why effective analytics is not simply about having more data or more sophisticated tools. It is also about being willing to challenge the question being asked.
This article is the second in our Beyond the Dashboard series, exploring how business intelligence, reporting and Artificial Intelligence are changing the way organisations interact with their data.
The Question Shapes the Analysis
Whenever someone investigates business performance, they usually start with a question. Why did revenue fall? Why are customers leaving? Why is one region outperforming another? Why did one product perform better this quarter?
Those are all sensible questions, but the challenge is that people often approach them with an existing theory. If sales have fallen, perhaps the assumption is that price is the problem. If one store is performing better than another, perhaps the difference is location. If customer churn has increased, perhaps service quality is immediately blamed.
Once that assumption exists, it becomes very easy to start seeking out information that supports it. The data may then be used to validate a theory rather than to investigate what is actually happening.
When a Convincing Chart Gives the Wrong Impression
This was demonstrated during a recent HAYNE Cognos Analytics session using retail store performance. An initial view suggested that recently renovated stores were performing better, which could easily lead to the conclusion that renovation was the main reason for improved results.
If that interpretation were accepted without further investigation, the business might decide that renovating more stores would lead to better performance. Deeper analysis, however, identified other factors within the data that could have a stronger relationship with the results.
The original chart was not necessarily incorrect. The issue was the conclusion being drawn from it.
Analytics can show that two things appear to move together, but that does not automatically prove that one caused the other.
Correlation Is Not the Same as Causation
This is one of the most important principles in analytics. Two measures can be correlated without one causing the other.
For example, a high-performing group of stores may also happen to share another characteristic. They could be located in areas with better accessibility, serve different customer demographics or operate with a different product mix. Looking at only one variable can create a convincing story, while looking more broadly may reveal that the situation has more complexity.
That does take away from the reliability of the dashboards or visualisations. It means they need to be interpreted carefully.
Business intelligence should help users explore a question and test possible explanations, rather than simply provide evidence for the answer they expected to find.
Bias Does Not Disappear Because You Have Data
Data is often described as objective, but the way people choose to examine it is not always. A user decides which measure to investigate, which time period to use, which filters to apply and which variables to compare. All of those choices can influence the analysis.
This creates the potential for confirmation bias. If someone already believes that a particular factor is responsible for a change in performance, they may naturally focus on the information that appears to support that view.
Modern analytics tools can help challenge this by making it easier to explore alternative explanations, identify unusual relationships and investigate areas that may not have been considered initially. The technology can support better analysis, but users still need to remain open to what the data is actually saying.
The Importance of Outliers
Another common mistake is focusing only on averages and overall trends. Outliers are sometimes treated as noise, when they can be one of the most useful parts of the analysis.
An unusually high or low result may point towards a data-quality issue, unusual customer behaviour, a process problem or something that requires deeper investigation. If every outlier is removed simply because it does not fit the expected pattern, potentially valuable information can disappear with it.
The same principle applies to positive results. If every KPI is green, that may look reassuring, but it can still be worth asking why. What is driving the improvement? Is the result sustainable? Is the metric still measuring the right thing?
Good analytics should encourage curiosity regardless of whether the number is good or bad.
Moving from “What Happened?” to “Why?”
Traditional reporting is particularly good at telling users what happened. Revenue fell, costs increased, a KPI missed its target or one region outperformed another. That is useful information, but it is usually only the starting point.
The real business value comes from understanding why.
That requires users to move beyond the surface-level number and investigate the factors behind it. This could mean looking at several dimensions of the data, comparing different groups, examining anomalies or challenging the original assumption altogether.
Increasingly, analytics platforms are helping with that process. Rather than requiring users to manually work through every possible variable, AI-assisted analytics can help identify patterns, relationships and possible drivers that deserve further investigation.
Can AI Help Us Ask Better Questions?
Artificial Intelligence introduces another development in how users interact with business data. Instead of opening a dashboard and applying a series of filters, users can increasingly start with a natural-language question such as:
Why did profitability fall last quarter?
The system can then help identify relevant information and surface possible explanations. This can make analytics more accessible, but it also creates a new responsibility.
A confident answer is not automatically the same as a correct one. The quality of the result still depends on the underlying data, how measures are defined and how the question itself is framed. If the source data is inconsistent, or the question contains an incorrect assumption, the resulting answer may still sound persuasive.
That is why human judgement remains necessary. The role of AI should be to help users explore the data more effectively, not to remove the need for critical thinking.
Better Questions Lead to Better Decisions
The goal of business intelligence should not simply be to produce more reports, dashboards or charts. It should help people make better decisions, and better decisions often begin with better questions.
Instead of asking “Can we prove our assumption?”, it may be more useful to ask, “What else could explain this result?”
Instead of asking “Which dashboard shows the answer?”, ask, “What information do we actually need to make the decision?”
And instead of assuming that a relationship within the data proves a cause, ask, “What other factors should we investigate?”
The best analytics environments do not simply give users answers. They help them challenge assumptions, explore alternatives and understand the story behind the number.
Because sometimes the biggest risk is not having the wrong data. It is asking the wrong question.
Want to Explore What This Looks Like in Practice?
If this topic is relevant to your organisation, we’ll be exploring it further in our upcoming webinar, Beyond the Dashboard: The Future of Cognos Analytics, Reporting and AI.
Join HAYNE Solutions and IBM for a practical look at how reporting, analytics and AI are evolving in Cognos Analytics, including a live demonstration of the latest capabilities.
Find out more and register for the webinar
Next in the Beyond the Dashboard series:
Why Data Governance Matters More in the Age of AI
In the next article, we’ll explore why trusted data, consistent definitions and strong governance become even more important as organisations introduce AI into reporting and analytics.
Previous article in the Beyond the Dashboard series:
Do You Really Need Another Dashboard?
