False balance

What it looks like

Two people appear. One represents a large body of evidence and one represents very little. The format says nothing about that, and the audience infers from the arrangement that the underlying question is open.

Nobody has to lie. That is what makes it durable: fact-checking each speaker individually returns nothing, because the misleading claim is not made by either of them. It is made by the layout.

Why it works

A content analysis of 636 articles sampled from four major US newspapers between 1988 and 2002 found that a majority — 52.65 percent — gave balanced treatment to the question of whether human activity was warming the planet, during a period when the scientific literature was not balanced on that question.1

That particular norm then largely collapsed in prestige print: a follow-up analysis of 9,465 articles found US balanced framing on the same question falling from 36.6 percent in 2003 to 3.3 percent in 2006.2 The imbalance did not disappear so much as move. A 2019 study tracking 386 prominent contrarians against 386 expert scientists across roughly 100,000 media articles found the contrarians appearing in 49 percent more articles overall, but only 1 percent more in mainstream sources.3

That the arrangement itself moves belief has been tested. In one experiment, participants shown balanced coverage of a claim with an unbalanced evidence base became less certain and more likely to believe experts were divided — and the effect ran specifically through perceived expert division.4 A second study found the distortion survived even when participants were also given a complete tally of how the full expert panel divided.5 Being told the real numbers did not undo what the format had already done.

Documented cases

Passive smoking in the US press, 1981–1994

Kennedy and Bero analysed 272 newspaper and magazine articles about research on second-hand smoke. Sixty-two percent concluded that the research was controversial, during a period when the evidence of harm was steadily accumulating.6

The detail that makes this a textbook case is what the second side actually consisted of. Tobacco industry representatives were quoted in 52 percent of newspaper articles. But of 121 distinct studies cited across the whole corpus, only 15 came from industry-funded work. The dissenting side was supplying criticism of other people’s methods, not competing evidence of its own — and the format presented that as a symmetrical disagreement.

A broadcaster changing its own rules

In July 2011 the BBC Trust published a review of impartiality and accuracy in the BBC’s science coverage, with an independent assessment by the geneticist Steve Jones, which criticised an over-rigid application of impartiality that gave undue attention to marginal opinion. A 2014 progress report restated the principle that impartiality "does not simply lie in reflecting a wide range of views" but depends on the due weight each view should be given.

In September 2018 the BBC issued internal reporting guidance stating that "as climate change is accepted as happening, you do not need a ‘denier’ to balance the debate."7 This is included as an example of an institution publicly identifying false balance in its own output and changing its editorial rule — the process, not a verdict on any dispute.

What to do about it

• Count the sides, then count the evidence behind each side. Two speakers is a production decision. Ask how many researchers and how many studies stand behind each one.
• Look for a weight-of-evidence sentence. A piece that tells you where the bulk of the evidence lies is doing its job. Its absence in something framed as a debate is the tell.
• Ask what the dissenting voice has actually published on the question. Criticising someone else’s method is not the same as having evidence, and the two look identical on a panel.
• Separate "is it happening?" from "what should we do?" Balance is appropriate on a values question and misleading on a settled empirical one. Many pieces mix them.
• Treat "the science is divided" as a claim that needs its own source. It usually does not have one.

Sources

Every figure on this page comes from one of these. Each entry names the publisher, the date and the sample size, so you can check it rather than take our word for it.

1. Boykoff, M. T. & Boykoff, J. M. — "Balance as bias: global warming and the US prestige press"Global Environmental Change 14(2):125–136 · 2004 · n=636 sampled from 3,543 articles, 1988–2002https://sciencepolicy.colorado.edu/admin/publication_files/2004.33.pdf
2. Boykoff, M. T. — "Flogging a dead norm? Newspaper coverage of anthropogenic climate change in the United States and United Kingdom from 2003 to 2006"Area 39(4):470–481 · 2007 · n=9,465 articleshttps://doi.org/10.1111/j.1475-4762.2007.00769.x
3. Petersen, A. M., Vincent, E. M. & Westerling, A. L. — "Discrepancy in scientific authority and media visibility of climate change scientists and contrarians"Nature Communications 10:3502 · 13 August 2019 · 386 vs 386 individuals, ~100,000 articleshttps://doi.org/10.1038/s41467-019-09959-4
4. Dixon, G. N. & Clarke, C. E. — "Heightening Uncertainty Around Certain Science"Science Communication 35(3):358–382 · June 2013 · n=327https://doi.org/10.1177/1075547012458290
5. Koehler, D. J. — "Can journalistic ‘false balance’ distort public perception of consensus in expert opinion?"Journal of Experimental Psychology: Applied 22(1):24–38 · 11 January 2016https://doi.org/10.1037/xap0000073
6. Kennedy, G. E. & Bero, L. A. — "Print media coverage of research on passive smoking"Tobacco Control 8(3):254–260 · 1999 · n=272 articles, 1981–1994https://doi.org/10.1136/tc.8.3.254
7. Hickman, L. — "Exclusive: BBC issues internal guidance on how to report climate change"Carbon Brief · 7 September 2018https://www.carbonbrief.org/exclusive-bbc-issues-internal-guidance-on-how-to-report-climate-change/
8. Hargreaves, I., Lewis, J. & Speers, T. — "Towards a Better Map: Science, the Public and the Media"Economic and Social Research Council · 20 May 2003 · 2,214 science stories analysed, 2 national surveys of 1,000+ respondentshttps://www.eurekalert.org/news-releases/524195