By now, “evidence-based decision making” has become one of those phrases that appears in almost every policy document, briefing note, and government slide deck.
I cringe when I see it or hear it.
It has become a moral label — a way of saying, “I am the rational one here.” People doing research start their roundtable discussions by proudly announcing that their research is ‘evidence-based’ as if somehow, other research or opinion pieces do not use evidence.
But increasingly, when someone insists they are using “evidence-based” approaches, I find myself thinking that if they have a mission statement with a social goal, they may be engaging in the opposite: what they may be really doing is decision-based evidence making (a phrase I attribute to Armine Yalnizyan and Richard Shillington). The decision is made early as part of the mission— sometimes emotionally, sometimes politically — and the evidence is then gathered afterward to justify it. Governments and political parties are the worst offenders.
The uncomfortable truth is that everyone uses evidence.
Policymakers, researchers, think-tanks, conspiracy theorists, and online grifters all accumulate “facts” that point toward the conclusion they already prefer. As Aristotle warned long ago and I paraphrase, what is attributable to everyone becomes attributable to no one. If everyone claims to use evidence, the word ‘evidence’ stops being meaningful.
The real distinction, then, is not between using evidence and not using it.
It is between using good evidence and bad evidence, and — just as important — between good reasoning and bad reasoning concerning that evidence.
To make progress, we need a clearer understanding of what counts as evidence, where it comes from, and the ways in which thinking can go wrong.
In my work, I organize data into six broad tiers and one is very lucky if they have access to all six. It seldom happens,
Each has strengths, limits, and appropriate uses:
- Big Data
Google clicks, metadata trails, platform scraping — vast, behavioural, often unstructured. - Government Non-Survey Data
Income Tax records, the Census, T1FF, LAD — high-coverage databases. - Survey Data
Everything from StatsCan surveys like the Canada Income Survey to private polling — structured but self-reported. - Administrative Program Data
Large operational spreadsheets, case files, service counts — essential but it often contradicts or is at odds with survey data. - Community-Based Research (CBR)
Probes, interviews, and community-grounded verification — invaluable for depth. - Lived-Experience Reportage
Transcripts, proceedings, unfiltered testimony — the texture behind the numbers.
This 6 part hierarchy of evidence matters because not all data can answer all questions.
Administrative data can tell you how many people received a benefit. Lived-experience accounts can tell you whether the benefit made any sense in the lives of recipients. Big Data can show patterns; CBR can explain them. Good evidence comes from knowing which tier matches which purpose.
Onto this data landscape we layer the five types of thinking that we all (not just researchers) routinely employ:
- Deductive
The axiomatic: 2 + 2 = 4. No observation required – 2+2 always = 4 - Inductive
The evidentiary: patterns, probabilities, everything from hunches to randomized trials. - Analogic
This is to this as that is to that — the bridge-building of human understanding. - Phenomenological
The inward and reductive deconstruction of the observed gestalt— stripping away assumptions meaning to reveal inner truths. - Dialectical
Position → negation → negation of the negation — the developmental logic of change.
Every important discipline uses all five. There are more but let’s leave it to these five for now.
Behavioural economics is a fusion of inductive trials, analogies to cognitive heuristics, phenomenological insight into human strain, and dialectical critiques of classical models.
The trouble begins when someone uses one of these forms of reasoning and mistakes it for the only form. Or uses them in an undisciplined way.
The lure of false certainty
Evidence becomes dangerous not when it is absent, but when it is over-confidently present.
This is the lure of false certainty:
- The survey with too few respondents that becomes a headline.
- The single anecdote that becomes a political rallying cry.
- The Big Data correlation mistaken for causation.
- The administrative outlier mistaken for a trend.
- The clever analogy mistaken for fact.
- The “my intuition tells me…” dressed up as insight.
Human beings resist uncertainty. Policymakers even more so. Uncertainty feels weak in public. Certainty feels strong.
This is why so much bad evidence thrives: it promises definitiveness long before definitiveness is possible.
And this is where grifters excel.
The grifter’s advantage: bad evidence, perfectly delivered
Modern grifters don’t peddle lies; they peddle misplaced confidence.
They offer:
- a simple dataset,
- a selective chart,
- a suggestive analogy,
- and above all, the tone of someone who “knows how it really works.”
They speak in the language of “research shows…” and “data proves…” while quietly choosing only the fragments that flatter their thesis. They exploit the gap between what data can show and what people wish it would or could show.
Bad evidence is rarely random. It is curated.
That is why conspiracy theorists always have “evidence” — sometimes pages and pages of it. It is why poor public policy can be defended with charts. It is why ideological positions come wrapped in the language of rigour.
What separates expertise from grift is not the presence of evidence, but the exquisite discipline of rejecting the evidence that does not fit.
Experts discard. Grifters decorate.
Good evidence requires:
- Choosing the correct data tier for the question;
- Naming the limits of the data as clearly as the findings;
- Using multiple modes of reasoning, not just the one that flatters your thesis;
- Allowing discomfort — letting the evidence unsettle you;
- Admitting uncertainty where uncertainty remains -and it always remains;
- Changing your mind when the weight of the evidence changes.
Decision-based evidence making does the opposite: It selects the data tier that supports the decision, the reasoning mode that best flatters the conclusion, and the narrative that conveys certainty without justification.
The discipline of good evidence is the discipline of resisting that temptation.
A better way to think about evidence
If we want to be truly evidence-based, we must give the word its meaning back:
- Evidence is tiered.
- Reasoning is varied.
- Certainty is rare.
- Doubt is healthy.
- And “I don’t know yet” is sometimes the most responsible finding of all.
We do not escape evidence; we are immersed in it. The question is not whether we use it, but how.
Good evidence humbles us. Bad evidence flatters us.
The former builds systems worth trusting. The latter builds stories that merely comfort their authors.
If we can teach ourselves — and others — to make this distinction, we may yet reclaim the phrase “evidence-based” from the realm of slogans and return it to where it belongs: the disciplined pursuit of what is true, rather than what is useful or moral.