What is Criterion validity?

Also called Criterion-related validity

Criterion validity is the strength of the relationship between a test score and an independent outcome the test is meant to anticipate — typically job performance, training results or retention. It is reported as a correlation coefficient, and it is measured either at the same time as the outcome or in advance of it.

Concurrent and predictive designs

In a concurrent study, current employees take the test and their scores are correlated with their existing performance ratings. It is fast and it is the design most validity evidence actually uses, but the sample has already survived selection, promotion and attrition, so its range is narrower than an applicant pool's.

In a predictive study, applicants take the test, everyone is hired regardless of score, and performance is measured later. It matches how the test is used, and it is expensive and rare for exactly that reason.

The criterion is usually the weak link

Validity coefficients are bounded by the reliability of the outcome measure. Supervisor performance ratings, the criterion in most published studies, are themselves noisy and contaminated by tenure, likeability and rating leniency.

So a modest correlation may reflect a weak test, a weak criterion, or both, and a study reporting a validity coefficient without reporting the criterion's own reliability has left out half of the information needed to read it.

Not the same as construct validity

Criterion validity asks whether the score predicts an outcome. Construct validity asks whether it measures the attribute claimed. A test can predict an outcome for reasons that have nothing to do with the construct it names.

Construct validity