What is Item response theory?

Also called IRT, Latent trait theory

Item response theory models the probability of a given answer as a function of the candidate's underlying ability and the question's own parameters — usually difficulty, discrimination and, for multiple choice, guessing. Because ability and item difficulty are placed on the same scale, scores from different sets of questions remain comparable.

What the models are

The one-parameter model, closely associated with the Rasch model, describes each item by difficulty alone. The two-parameter model adds discrimination, so items differ in how sharply they separate people. The three-parameter model adds a lower asymptote for guessing, which matters for multiple-choice items where a candidate who knows nothing still has a floor.

Choosing among them is a modelling decision with consequences: more parameters fit better and need substantially more data to estimate stably.

What it buys, and what it costs

The payoff is invariance. Once items are calibrated onto a common scale, two candidates who saw different questions can be compared, new items can be added to a bank without re-standardising the whole test, and the next item can be selected adaptively to maximise information at the candidate's current estimate.

The cost is data and discipline. Calibration needs large samples per item, the model's assumptions — chiefly that a single dimension accounts for the responses, and that answering one item does not change the odds on another — need checking rather than assuming, and a bank calibrated on one population may not hold on another.

Not the same as classical test theory

Classical test theory works with total scores and treats item statistics as properties of a sample. Item response theory models each response individually and places people and items on one scale. Most commercial assessments still report classical statistics; the ones that support adaptive delivery cannot.