Before You Put It in the Deck: Data and Analytics Judgement

Somebody sends you a finding. Tickets are up forty per cent. Conversion fell after the redesign. One branch is beating the network. In two hours it will be on a slide, and the only person who can say what it is worth is you. This assessment is forty written exercises inside that moment, and none of them is about a tool. There is no query to write, no formula to recall and no arithmetic beyond reading a percentage. Six exercises ask whether a difference is real or is the ordinary swing of a small number. Six ask what a figure actually covers before it is quoted: what set of rows, what date, whose definition, and who is missing from the data entirely. Eight describe a chart in words, together with the sentence somebody wants to put under it, and ask what is true: the drawing distorts, the drawing is fair and the sentence claims more than it shows, or both are fair and the feature that looks wrong is legitimate. Six put you in the room with somebody who wants the number to be different. The largest part is fourteen exercises across four fully written scenes. Each scene states a pattern and the evidence you actually hold, then names the competing explanations one at a time and asks how much weight the evidence gives each. Half of the evidence in every scene is qualitative, so nobody is scored on arithmetic. The scoring is a centred-contrast allocation match rather than a mark out of forty. The keyed weights in each scene are centred on their own mean, so an explanation the evidence backs above average carries a positive weight and one it backs below average carries a negative one. Your score is the weight you gave, multiplied by that centred key, added up. Two habits are worth exactly nothing by construction. Spreading the same weight across every explanation scores zero in every scene, because the centred key sums to zero. Putting all of it on the loudest explanation also scores zero across the four scenes, because the loudest explanation is keyed decisive in one scene, unsupported in another, and in the middle in the rest. Those are two different errors and the report names them separately: spread thin, and first-story lock. Your score is expressed against what a respondent answering at the declared rates would score, so zero means typical rather than nothing right. The report is drawn as four explanation boards, one per scene, with each explanation standing above or below the line by how much the evidence backs it and your own weight drawn on it, so you can see where the number came from. It measures what a person does with a finding before it is quoted: whether a difference is real, what a metric covers, how much weight each competing explanation deserves, when a chart is honest and when it is persuasion, and what to say when somebody wants the number to be different. It does not measure query writing, statistical technique, tool knowledge, arithmetic, speed, or how good the person is at the job around the analysis.

Bought this already and lost the tab? Sign in and enter your purchase code under Claim a purchase on your dashboard.