Reading Statistics in News, Advertising and Social Media Assessment for Everyday ReadersHalf of these numbers are honest. Which ones would you have forwarded, and which would you have thrown away?
Twenty-five short claims of the kind that arrive in a feed — a headline, an advertisement line, a product page, a post a friend sent — each with a number under it and one question: does the number support what it is being used to say? Twelve of them are sound. Scored as two figures that are never added, plotted as one point on a quadrant, with the reader who distrusts everything drawn on the same chart.
A test that prices suspicion instead of rewarding it
The Reading Statistics in News, Advertising and Social Media Assessment for Everyday Readers is a twenty-minute check of whether a person can tell when a number supports the claim attached to it, across twenty-five invented headlines, advertisements and posts, half of them sound, reported as two figures that are never combined.
Most quizzes about misleading statistics show you a stack of tricks and ask you to spot them. A reader who calls everything misleading scores full marks on such a quiz, and learns nothing except to distrust numbers. Here twelve of the twenty-five claims are sound: a leaflet that gives the risk with and without screening out of the same thousand people, a survey drawn at random with most of those approached taking part, a comparison done by lot. Rejecting those is counted, printed, and named as the habit it is.
Every claim carries a declared prior, the share of ordinary readers expected to call it correctly. A subtle claim, one that seven in ten readers misread, counts for more than an obvious one, and the standard deviation of those priors is printed on the report because they are the scale. Zero on each figure is the ordinary reader, not half marks: a score above zero means you read these claims better than people typically do, and a score below it means the claims got through more often than they usually get through.
The two figures, caught and kept, are plotted as one point on a quadrant, with its 68 and 95 per cent bands drawn as bars on both axes and written in words beside the drawing. Each of the four cells is labelled with an action rather than an adjective: find the denominator before you forward, slow down before you dismiss, keep reading this way, one check on the next three numbers. Where a band crosses a line the page says the cell is not settled, and the two fixed habits, call everything misleading and call everything sound, are priced on the sitting's own claims and drawn as hollow diamonds on the same chart.
On a single-choice claim the reader has to name the flaw, not just flag the claim: a percentage-point rise called a percentage, an average carried by a few large values, a sample that chose itself, only the survivors counted, a start date chosen to make the trend, a change inside the margin of error. A claim flagged under the wrong flaw is printed as such and not counted as caught, so the report can say which named flaw you miss most, and, on the sound claims you rejected, which objection you reached for. That last strip is the one people send to the friend who forwarded the number.
Nothing here touches a party, a candidate, an election, a religion or a nation. The claims are about health, money, products, local services, schools, weather and sport, all invented, with no real brand, publication or person named. The report says in plain words what it did not measure: intelligence, numeracy beyond simple arithmetic, general suspicion, or whether you are easily fooled. It measured how you read twenty-five short claims on one afternoon, and it prints the priors, the reliability and the refusals alongside the figures.
What you walk away with
Weighted by how subtle each claim was and corrected against the ordinary reader, with its 68 and 95 per cent bands. On a single-choice claim, only the reading that names the actual flaw counts.
The same arithmetic on the twelve honest claims. Calling everything misleading scores near the top on caught and near the bottom on kept, and the report draws that reader on the quadrant so the habit is priced rather than rewarded.
A risk halved from 40 to 20 in 10,000; a charge that rose by one point from 2 to 3 per cent; an average of 9 lakh where half the graduates earn under 3.5; more injuries on a lane carrying three times the cyclists.
A survey of customers who kept the product and chose to reply; the firms that survived fifty years and never borrowed; the share of the injured in socks with no share of everybody given; and a random sample that is exactly as good as it says.
A fall measured from the peak month; two averages of two different groups of customers; children who eat at a table and read better; and a bus-pass trial done by lot that earned its claim.
Two points inside a three-and-a-half-point margin; a pass rate that rose eight points because one student of twelve passed; the three worst branches improving after training; and a fall at a junction that clears eight years of ordinary swings.
Inside your report
Illustrative sample — your report is generated from your own responses.
How to read it: the dot is you; the thick bar in each direction is the 68 per cent band and the thin bar with end ticks is the 95 per cent band. Across: two chances in three between 27 and 49, nineteen in twenty between 17 and 59. Up: two chances in three between -25 and -3. The band crosses the kept line, so the page says the cell is not settled. The hollow diamonds are the two habits, priced on the sitting's own claims: distrust everything lands bottom-right, not top-right.
| Point | Caught | 95% band | Kept | 95% band |
|---|---|---|---|---|
| ● you | 38 | 17 to 59 | -14 | -36 to 8 |
| ◇ calls everything misleading | 86 | — | -92 | — |
| ◇ calls everything sound | -56 | — | 100 | — |
On the 3 sound claims you rejected, the objection you reached for most was: too few people to trust the figure (2 times). That is the suspicion to slow down on.
| Named flaw | Caught | Wrong flaw | Read as sound |
|---|---|---|---|
| ○ Relative change where the absolute change is tiny | 0 | 0 | 1 |
| ● A percentage-point change called a percentage | 1 | 0 | 0 |
| ◆ Only the survivors were counted | 0 | 1 | 0 |
| ● The share among everybody was never given | 1 | 0 | 0 |
| ● Two things moving together read as a cause | 1 | 0 | 0 |
| ○ A change inside the margin of error | 0 | 0 | 1 |
A claim flagged under the wrong flaw is printed as such and is not counted as caught: the caught figure is a reading figure, not a suspicion figure.
Built for
- Anybody who reads a number in a headline, an advertisement or a post and wants to know whether they would have forwarded it
- Students and early-career readers building the habit of asking what a number is out of, before an exam or a first job asks them
- Journalists, communicators, marketers and campaigners who put numbers in front of other people and want to know which flaws slip past them
- Parents, teachers and group admins who are sent things, and would like a report they can send back
Find out what you catch, and what you throw away
36 exercises across five formats · about 20 minutes · two figures on a quadrant, never added, with the reader who distrusts everything drawn beside you, and the named flaw you miss most.
Take free · full report ₹249 (incl. GST)
Frequently asked questions
Because without them the winning strategy would be to call everything misleading, and the test would measure suspicion rather than reading. Twelve of the twenty-five claims use their number honestly. Rejecting one of those is counted on its own figure, kept, which is printed beside caught and never added to it. The report also draws the reader who distrusts everything on the quadrant, so you can see where that habit lands.
Zero is the ordinary reader, not half marks and not an empty sitting. Every claim carries a declared prior, the share of ordinary readers expected to call it correctly, authored per option in the open. A subtle claim counts for more than an obvious one, and each figure is corrected against what the ordinary reader would score with the same weights. The priors are assumptions that observed answers will replace, and the report says so where it uses them.
No. No claim is about a party, a candidate, an election, a religion or a nation, and no answer depends on which side of anything you are on. The claims are about health, money, products, local services, schools, weather and sport, all invented, with no real brand, publication or person named. The report's refusal block says this in plain words.
On a single-choice claim you choose one of four readings, and only the reading that names the actual flaw counts as caught. A claim flagged under a different flaw is printed as flagged under the wrong flaw and is not credited, so the caught figure is a reading figure rather than a suspicion figure. The report then lists which named flaw you miss most and, on the sound claims you rejected, which objection you reached for.
About twenty minutes for thirty-six exercises across five formats. The sitting is free to take; the report is the product, priced at ₹249 in India, inclusive of GST, or US$2.99 elsewhere, one time. The report is written to be read by the person who forwarded you the number: two figures on a quadrant, four cells each labelled with an action, the named flaw you miss most, one if-then rule in your own first person, and what the test did not measure.
Each one takes a single capability, puts you inside the situations where it is actually tested, and scores your choices against published evidence — with a report designed for that capability alone, not a template. They span hiring, compliance, education, operations and personal skill.
Browse the catalogue →Methodology: Thirty-six original exercises across five formats: sixteen single-choice claims, each a headline, advertisement line, product page or post with a number under it and four readings of it, one keyed; nine true-or-false claims of the same kind; five select-every-that-applies exercises; three match-the-following exercises; and three numeric estimation exercises on a slider. One response instruction is declared for the whole instrument and it is a KNOWLEDGE instruction: which reading of this number is right, never what the respondent would do or feel. Construct statement: this measures how well a person tells whether a number supports the claim it is attached to, in everyday news, advertising and social media, and what they know about the named ways numbers go wrong; it does not measure intelligence, numeracy beyond simple arithmetic, general suspicion, political views, or whether the person is easily fooled, and nobody is described anywhere on the report as gullible, naive or careless. Scoring is asymmetric-lure detection with a declared take-up prior (C22). Every claim is classed as a lure, where the number does not support the claim, or genuine, where it does; twelve of the twenty-five are genuine by design, because without that balance calling everything misleading would be the winning strategy and the instrument would measure suspicion rather than reading. Each claim carries a declared prior: the share of ordinary readers expected to call it correctly, authored per option and per side. Each claim is weighted by one minus its prior, so a subtle claim counts for more than an obvious one, and the standard deviation of the declared priors across the twenty-five claims is printed in the seed and on the report because the priors are the scale: without dispersion the weighting collapses into plain accuracy. Two figures are computed and never combined: caught, the weighted share of lures read correctly, and kept, the weighted share of genuine claims not wrongly rejected. On a single-choice lure, caught credits only the reading that names the actual flaw; a reader who flags the claim under a different flaw is recorded as having flagged it for the wrong reason and is not credited, and the report prints those separately. Each figure is corrected against what an ordinary reader would score with the same weights, so zero means no better than the way people typically read these claims, and each carries its standard error band from an assumed omega that is printed unrounded. A figure is printed only when at least eight claims of that class were answered and omega is at or above .70 unrounded; otherwise a three-way placement is printed with the refusal in the figure's place. A sitting with fewer than twenty answered exercises is not reported: the quadrant is withheld and the refusal is printed where it would have been. An unanswered exercise leaves the numerator, the denominator and the chance term together; an empty sitting scores exactly zero on both figures. The report also prints what a reader who calls everything misleading and a reader who calls everything sound would score on this sitting, as two reference points drawn on the same quadrant, so that either habit is priced rather than rewarded. The knowledge exercises are corrected against the declared per-option, per-pair and per-value priors and reported per family as a three-way placement, because no family carries eight of them. Every declared prior is an authored assumption stated in the open and will be replaced by observed answer shares once live data exist. A careless-responding flag count is computed for the operator and never shown to the respondent as a judgement. No exercise touches a party, a candidate, an election, a religion or a nation; the claims are about health, money, products, local services, schools, weather and sport, and every one is invented with no real brand, publication, company, person or place named. Sources drawn on: Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz and Woloshin, Helping doctors and patients make sense of health statistics (2007); Gigerenzer and Hoffrage, How to improve Bayesian reasoning without instruction: frequency formats (1995); Schwartz, Woloshin, Black and Welch, The role of numeracy in understanding the benefit of screening mammography (1997); Tversky and Kahneman, Judgment under uncertainty: heuristics and biases (1974); Kahneman and Tversky, On the psychology of prediction (1973), for regression to the mean; Wainer, Visual revelations (1997) and Picturing the uncertain world (2009), for the small-numbers and denominator lessons; Huff, How to Lie with Statistics (1954); Best, Damned Lies and Statistics (2001); Spiegelhalter, The Art of Statistics (2019); Wason, On the failure to eliminate hypotheses in a conceptual task (1960) and Pennycook and Rand, Lazy, not biased: susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning (2019), for the reading-versus-suspicion distinction; Macmillan and Creelman, Detection Theory: A User's Guide (2005), for the two-error framing; Gollwitzer and Sheeran, Implementation intentions and goal achievement, a meta-analysis (2006), for the if-then action; and Haladyna, Downing and Rodriguez (2002) for the item-writing rules. All exercises are original works written for this instrument. This instrument is not affiliated with, endorsed by or derived from any commercial instrument, certification, curriculum or publisher, and the flaw vocabulary it uses is the public vocabulary of statistical literacy.