What is Validity?

Validity is not a property of a test. It is the degree to which evidence and theory support the interpretations made of test scores for a specific proposed use. The same instrument can be valid for one decision and invalid for another, so validity is always stated in relation to a purpose.

The unitary view, and why it changed the question

Older writing treated validity as a set of separate types — content, criterion, construct — that an instrument could possess. The current professional standards treat it as a single concept supported by different sources of evidence: evidence based on test content, on response processes, on internal structure, on relations to other variables, and on the consequences of testing.

The practical effect is that 'is this test valid?' is not a well-formed question. 'What evidence supports using this test to decide X about this population?' is, and it is answerable.

What to ask for

Evidence for your use and your population, not a general claim. A test validated for graduate selection in one country is not thereby validated for frontline hiring in another, and translation is not validation.

Ask also about consequences. A procedure can predict adequately and still produce outcomes the organisation would not defend — which is why fairness evidence sits inside the validity argument rather than beside it.

The second half of the question: valid for this decision, and additional to what?

Even a well-formed validity question is only half a purchasing question. You are not choosing between a test and nothing; you are choosing between your current process and your current process plus this test. The figure that answers that is incremental validity, and it is always smaller than the coefficient on the data sheet, because two procedures that both predict performance usually overlap.

How much smaller is no longer a theoretical worry. Sackett, Zhang, Berry and Lievens report that in their illustrative composite, giving cognitive ability a weight of zero reduces validity by 0.05 — from 0.61 to 0.56 — despite ability having been treated for twenty-five years as the anchor predictor every other instrument had to justify itself against. A predictor can be genuinely valid and still be nearly redundant next to what you already run.

So the question to put to any vendor has two clauses. Valid for what decision, about which population — and additional to what we already do.

The demonstration: one dataset, four published validities, no new evidence

There is a way to see the unitary view working that needs no theory at all, only arithmetic. Almost every validity coefficient on a vendor data sheet has been corrected for criterion unreliability — the job-performance rating it was validated against is itself measured with error, so the observed correlation is divided by the square root of that rating's reliability. The correction is legitimate and standard. What is rarely visible is that the author chooses which reliability figure to divide by, and the literature currently offers several.

Take an observed correlation of 0.31 between a test and a supervisor rating, and correct it four ways with figures that have all been used in print. Corrected with 0.45, the administrative grand mean this site itself argued from until last week: 0.31 divided by the square root of 0.45 gives 0.46. Corrected with 0.52: 0.43. Corrected with 0.65, the overall figure Zhou, Sackett, Shen and Beatty report from 132 independent samples in 2024: 0.38. Corrected with 0.57, the same paper's figure for managerial positions: 0.41.

Four numbers between 0.38 and 0.46 out of one dataset, a spread of 0.08, which is about a fifth of the smallest of them. Nothing about the test changed. Nothing about the candidates changed. The only thing that changed was a decision about the criterion, made by whoever wrote the data sheet, and in most cases not disclosed on it.

That is what it means to say validity is not a property of a test, stated as something a buyer can check rather than as a principle. The operational question is one line: which criterion reliability was this coefficient corrected with, and is that figure disclosed? A vendor who cannot answer has handed you a number whose second decimal place is a choice — and the 2024 paper's own advice, against using an overall grand mean at all because the figure differs by job family, means the honest answer is now longer than it used to be.

Correcting one observed correlation four ways

Observed correlation between test and supervisor rating: r = 0.31.
Corrected r = r divided by the square root of the criterion's reliability.
With 0.45 the corrected figure is 0.46. With 0.52 it is 0.43. With 0.57 it is 0.41. With 0.65 it is 0.38.

The 0.31 is an illustration, not a figure from an AssessAll instrument — AssessAll publishes no validity coefficient for any of its own instruments, because none has yet passed its internal research-readiness gate. The four divisors are real: 0.45 and 0.52 are figures in long circulation, and 0.65 overall and 0.57 for managerial positions are what the 2024 cumulation reports. Run the same division on any coefficient a vendor hands you.

Not the same as reliability

Reliability asks whether the measurement is consistent. Validity asks whether the conclusion drawn from it is justified. Reliability is necessary for validity and nowhere near sufficient.

Reliability

Sources

Check a selection process against the four-fifths rule

Free, no signup, computed in your browser — with the remedy, not just the verdict.

Open the calculator

Last reviewed 2026-09-28