More Data Doesn't Always Mean a Better Decision

When I first started learning data analysis, I assumed that more data would naturally lead to better answers.

More rows.

More variables.

More detail.

Surely, the more information I had, the more confident I could be in the conclusion.

I don't think that anymore.

Sometimes more data improves an analysis.

Sometimes it just gives you more things to look at.

And those are not necessarily the same thing.

The temptation to analyse everything

Imagine sales have dropped.

You have customer data, product data, transaction history, discounts, locations, sales channels, marketing activity and probably another spreadsheet somewhere that nobody has opened since 2023.

It is very tempting to analyse all of it.

Break sales down by region.

Then product.

Then customer segment.

Then month.

Then channel.

Then age group.

Then perhaps create another five measures because Power BI is already open anyway.

Before long, one relatively simple business question has turned into a very impressive collection of charts.

But has the decision actually become any clearer?

That's the part I find more interesting.

Because the purpose of analysis isn't to prove how much analysis we can do.

It is to help someone understand what is happening and decide what to do next.

When does another piece of information actually matter?

Let's say sales are down 12%.

After looking at the data, we discover that most of the decline comes from one product category.

We investigate further and find that the change started at roughly the same time as a pricing change.

We also see that new customer sales are relatively stable, while repeat purchases in that category have fallen significantly.

At this point, there are still hundreds of things we could analyse.

But there is another question worth asking:

Would knowing them change what we do next?

Maybe the business already has enough evidence to review the pricing change, investigate customer behaviour in that category and decide what needs testing next.

That doesn't mean the analysis is finished forever.

It means there may be enough information to make the next decision.

And I think that distinction matters.

Having data doesn't mean I have to use it

I came across this recently while analysing my own job search.

I had tracked almost 300 applications in Excel and wanted to understand what the data could tell me about where my search was breaking down.

There were plenty of variables I could analyse.

One of them was whether roles were remote, hybrid or office-based.

It sounded useful.

The problem was that I hadn't recorded it consistently enough.

Some job adverts were very clear.

Others were vague.

Some changed during the recruitment process.

And in earlier applications, I hadn't always captured the information at all.

Technically, the column existed.

That didn't mean I trusted it enough to build a conclusion around it.

So I left it out.

It felt slightly strange at first.

If I've collected data, surely I should do something with it?

But having data doesn't automatically make it useful data.

Sometimes deciding not to analyse something is part of the analysis too.

More detail can make the important thing harder to see

There is another problem with analysing everything.

Important findings can disappear inside the detail.

If a dashboard contains 20 KPIs, 14 charts, six filters and three pages of drill-downs, technically it may contain a lot of information.

But what should I notice first?

What has changed?

What needs attention?

What decision does this information support?

Adding another visual is easy.

Deciding that it doesn't deserve space is much harder.

The same applies beyond dashboards.

A very detailed analysis can create the feeling that we are being thorough, while making the actual business problem harder to see.

Sometimes the strongest insight in an analysis is surprisingly simple.

It doesn't become more valuable because we surround it with another ten findings.

There is also a risk of false confidence

More data can make an analysis feel more certain.

But quantity and quality are different things.

Ten unreliable variables don't necessarily tell me more than three well-understood ones.

A huge dataset doesn't automatically answer the right question.

And a precise number isn't necessarily a useful number.

If the underlying data is incomplete, inconsistently defined or collected for a completely different purpose, analysing it in greater detail doesn't magically fix those problems.

Sometimes it just produces a more precise-looking version of uncertainty.

That is something I try to remind myself of.

The question I now try to ask

When I notice myself disappearing down another analytical rabbit hole, I try to come back to one question:

Would this change the decision?

If the answer is yes, keep investigating.

If the answer is maybe, understand what uncertainty needs resolving.

If the answer is no, perhaps the analysis doesn't need another chart, variable or calculation.

Of course, real business decisions aren't always that simple. Sometimes more evidence is needed because the cost or risk of getting something wrong is high.

But that's exactly the point.

The amount of analysis should be driven by the decision and its consequences - not simply by how much data happens to be available.

Final thought

One of the things I'm learning about data analysis is that knowing what to analyse is only part of the job.

Knowing what not to analyse matters too.

More data can provide more context, challenge assumptions and uncover things we didn't expect.

But more isn't automatically better.

At some point, another breakdown, another variable or another chart stops improving the decision and starts adding noise.

Good analysis isn't about extracting everything the data can possibly tell us.

Sometimes it's about knowing when we already know enough to make a better decision.