Before You Put It in the Deck: Data and Analytics JudgementA finding is only worth what the analyst does with it in the next two hours.
Forty written exercises inside that moment, and not one of them asks you to write a query. Whether a difference is real, what a metric actually covers, how much weight each competing explanation deserves, when a chart is honest and when it is persuasion, and what to say to somebody who wants the number to be different.
In brief
Before You Put It in the Deck: Data and Analytics Judgement is an online assessment in AssessAll's Applied Judgment Assessment line, with 40 scored exercises taking about 45 minutes. It measures is the difference real, what the number actually covers, how much weight each explanation deserves and when a chart stops being a picture, plus 1 more.
| Format | Online, one sitting |
|---|---|
| Duration | About 45 minutes |
| Scored exercises | 40 |
| What it measures | Is the Difference Real, What the Number Actually Covers, How Much Weight Each Explanation Deserves, When a Chart Stops Being a Picture, When Somebody Wants a Different Number |
| Built for | Employers hiring analysts, business intelligence and reporting staff, where the risk is a confident wrong reading rather than a broken query; Analytics and data leads who want to know which of their team over-reads noise and which lets a misleading chart through; Analysts and consultants who can already write the query and want an honest, private read on the judgement around it |
| Scoring | Evidence-keyed |
| Report | Available immediately after submission |
| Currencies | INR and USD |
It tests judgement, and it publishes the arithmetic behind the score
Almost every analytics test on the market checks whether you can produce a number. Write the query, fix the formula, read the pivot. That is a real skill and it is not the one that decides whether an analysis helps or misleads. The decisive moment comes after the number exists: somebody sends you a finding, tickets are up forty per cent, conversion fell after the redesign, one branch is beating the network, and in two hours it will be on a slide. This measures what you do in those two hours. There is no tool to know and no arithmetic beyond reading a percentage.
The largest part is four fully written scenes. Each states a pattern and the evidence you actually hold, then names the competing explanations one at a time and asks how much weight the evidence gives each. Half the evidence in every scene is qualitative, so nobody is scored on numeracy dressed up as judgement. Eight more exercises describe a chart in words, together with the sentence somebody wants under it, and ask what is true: the drawing distorts it, the drawing is fair and the sentence claims more than it shows, or both are fair and the odd-looking feature is legitimate.
The scoring makes two opposite habits worth nothing, and it can prove it rather than hope it. The keyed weights in a scene are centred on their own mean, so giving every explanation the same weight scores exactly zero, at any level, as arithmetic rather than as a threshold somebody chose. The loudest explanation is keyed decisive in one scene and unsupported in another, so putting all of it on the vivid cause scores zero as well. Those two land on the same figure and are opposite errors, and the report names them separately: spread thin, and first-story lock.
What you walk away with
One board per scene. Explanations the evidence backs sit above the rule, ones it does not sit below, and your own weight is drawn on the same signed axis. You can add the columns up by eye and get your own score.
Spreading weight evenly and putting it all on the loudest explanation score the same, by design. A concentration reading beside the score separates them, with the keyed allocation marked on the same rail, so you learn which mistake is yours.
Chart misreadings are split by direction and reported apart from the score. Marking honest work as manipulation and letting a crooked chart into the deck are both errors, they are not the same error, and one of them costs you a colleague.
The headline is corrected against authored per-option answer rates rather than a uniform guess, so zero is the typical respondent rather than an empty page. The raw figure, the chance figure and every declared rate are printed on the page.
Every reported figure carries its standard error at the sixty-eight and ninety-five per cent spans, drawn and written out. Four of the five parts carry no number at all, and the reliability table that justifies withholding them is printed rather than the refusal being made in silence.
Inside your report
Illustrative sample — your report is generated from your own responses.
E1 was the story everybody arrived with and the evidence barely supports it. A bar above the rule earned; a bar below it cost. The picture is not a decoration of the score, it is the score, one multiplication per column.
Built for
- Employers hiring analysts, business intelligence and reporting staff, where the risk is a confident wrong reading rather than a broken query
- Analytics and data leads who want to know which of their team over-reads noise and which lets a misleading chart through
- Analysts and consultants who can already write the query and want an honest, private read on the judgement around it
- Product, finance and operations managers who commission numbers and have to decide how much weight to put on what comes back
Somebody is going to act on that number either way.
Forty-five minutes, one sitting, the full report immediately — including the arithmetic behind every figure on it.
₹1,499 (incl. GST) · assessment and full report, nothing further to pay
Frequently asked questions
What an analyst does with a finding before it is quoted. Five parts: whether a difference is real or ordinary variation, what a figure actually covers, how much weight each competing explanation deserves, when a chart is honest and when it is persuasion, and what is said to somebody who wants the number to be different. It does not test SQL, spreadsheet technique or any tool.
The largest part scores your weight against an authored expert allocation that is centred on its own mean. That makes two habits worth nothing as arithmetic rather than as policy: giving every explanation the same weight scores exactly zero, and putting it all on the loudest explanation scores zero too. On the chart exercises each of the three possible readings is keyed on about a quarter of them, so no fixed answer pays.
Forty-five minutes in one sitting, forty written exercises, nothing to install and no software to open. The full report arrives immediately: the explanation boards, the ledger index with both of its error bands drawn, the concentration reading that separates the two opposite errors, the chart misreading split, the reliability table and one if-then action. There is nothing further to unlock.
Hiring teams read the two directional readings rather than the headline, because they answer the question a score cannot: does this person over-read noise, or let a misleading chart past. Individuals read the boards, which show where every point came from, and the single if-then action at the end. A re-measurement threshold is printed, so a second sitting can be judged honestly.
Yes. The scenes are ordinary business situations — a support queue, a checkout redesign, a branch network, a short test — and none of them needs local knowledge to answer. The evidence is written out in full and the difficulty lives in the judgement rather than the vocabulary. It is priced in INR and USD, and organisations can use AssessAll credits instead.
Each one takes a single capability, puts you inside the situations where it is actually tested, and scores your choices against published evidence — with a report designed for that capability alone, not a template. They span hiring, compliance, education, operations and personal skill.
Browse the catalogue →Methodology: Measures applied data and analytics judgement through original items keyed to published constructs. Construct statement: it measures what a person does with a finding before it is quoted — whether a difference is real or is ordinary variation, what a stated figure actually covers, how much weight each competing explanation deserves on the evidence held, when a chart is an honest picture and when it is persuasion, and what is said to a stakeholder who wants the number to be different. It does not measure query writing, statistical technique, tool or software knowledge, arithmetic beyond reading a percentage, speed of work, general intelligence, or performance in the job around the analysis. Constructs and published sources: common-cause against special-cause variation and the cost of reacting to noise as though it were signal, from statistical process control (Shewhart; Deming's distinction between the two kinds of variation); regression towards the mean, and why selecting the worst performers and measuring again produces apparent improvement (Galton; Kahneman and Tversky on the illusion of the treatment effect); multiplicity and the expected number of findings when many comparisons are run (Benjamini and Hochberg on false discovery; the published literature on garden-of-forking-paths analysis, Gelman and Loken); the base-rate and small-numbers problem, and the law of small numbers (Tversky and Kahneman); construct validity and proxy measurement, where an indicator is quoted as though it were the thing it stands for (Campbell's law on the corruption of an indicator once it becomes a target; Goodhart's law as stated for measures under pressure); denominator neglect and the reference-class problem in rate reporting; non-response bias and the difference between the sample that answered and the population that was asked (survey methodology literature on non-response error); explanation weighting under competing hypotheses, from the analysis-of-competing-hypotheses method published by the United States Central Intelligence Agency's Center for the Study of Intelligence (Heuer, Psychology of Intelligence Analysis, chapter 8), which asks the analyst to weigh evidence against every hypothesis rather than to build a case for the leading one; availability and the first-story effect, in which the most retrievable explanation is treated as the most probable (Tversky and Kahneman); confirmation bias and the selective search for supporting evidence; graphical integrity, the lie factor and the truncated baseline, from the published literature on statistical graphics (Tufte, The Visual Display of Quantitative Information; Huff, How to Lie with Statistics; Wainer on display of data); area against length encoding of a linear quantity; the cumulative-total display and why a running total cannot show a change in rate; the part-period comparison; correlation presented with causal language over observational data; implementation intentions as the mechanism behind a single if-then action (Gollwitzer and Sheeran, meta-analysis across 94 studies); McDonald's omega as the reliability estimator in place of coefficient alpha (McDonald; Revelle and Zinbarg); the standard error of measurement and the reporting of a band rather than a point (Crocker and Algina); and the reliable change index as the threshold at which a second sitting differs by more than measurement error (Jacobson and Truax). Declared response instruction, stated per part rather than claimed uniformly: four of the five parts ask what is true of the evidence written in front of the respondent — a knowledge and judgement framing — and one part, on what is said to a stakeholder who wants a different number, asks what the respondent is most likely to do, which is a behavioural-tendency framing. The two are never mixed inside one scale, each part is scored on its own, and the report says which part is which and that the behavioural part is the most fakeable of the five. Item format: an evidence ledger with a fixed weight budget, delivered as one item per competing explanation, combined with a red-pen review of charts described in prose. Every scene is written out in full, is identical for every respondent, and does not respond to anything chosen earlier; half the evidence in every scene is qualitative, so numeracy is not a confound. Scoring design: centred-contrast allocation match with a dispersion co-read. The keyed weights within a scene are centred on their own mean, the respondent's weight on each explanation is multiplied by that centred key, and the result is summed and expressed against the expected value under authored per-option base rates. Because the centred key of a scene sums to zero, any constant allocation scores exactly zero; because the centred key of the most salient explanation sums to zero across the four scenes, any all-on-the-loudest allocation also scores zero. Both are named in the report as distinct errors and separated by the spread reading. Reliability is stated as an assumption from item count and an assumed average inter-item correlation until live data exists, no competency under eight items carries a number, no percentile is claimed anywhere, and no norm group exists or is implied. All items are original works. No commercial instrument, vendor, branded methodology, software product or trademarked construct vocabulary is reproduced or named, and no affiliation with any source is claimed. AssessAll original design.
Last updated