--- title: "Item analysis" description: "Item analysis is the statistical review of how each question in a test performed. Two numbers carry most of it: difficulty, the proportion of candidates who answered correctly, and discrimination, the correlation between getting that question right and scoring well on the test overall. Weak items are revised or retired on this evidence." canonical: https://www.assessall.com/guides/glossary/i/item-analysis updated: 2026-09-26 source: AssessAll --- # Item analysis _Also known as: Classical item statistics._ Item analysis is the statistical review of how each question in a test performed. Two numbers carry most of it: difficulty, the proportion of candidates who answered correctly, and discrimination, the correlation between getting that question right and scoring well on the test overall. Weak items are revised or retired on this evidence. ## Difficulty, and why the extremes are wasted Difficulty is conventionally written as p, the proportion correct — so a high p means an easy item, which reads backwards until you are used to it. An item everybody answers correctly and an item nobody answers correctly both carry no information about differences between candidates, however good the question looks. For a test intended to spread people out, items around the middle of the range do most of the work. For a test intended to certify against a fixed standard, the useful items are those near the cut score, which is a different target and implies a different bank. ## Discrimination is the number that finds broken items Discrimination is usually the point-biserial correlation between the item and the total score. A value near zero means the item is unrelated to whatever the rest of the test measures. A negative value means strong candidates are getting it wrong more often than weak ones — almost always a miskeyed answer, an ambiguous stem, or two defensible options. A common working rule is to review anything below about 0.20 and to treat any negative value as a defect until proven otherwise. Both figures depend on the sample, so they are unstable on small numbers of responses and should not be acted on before a bank has accumulated enough sittings. ## How many sittings before either number means anything This entry has asserted for weeks that item statistics are unstable on small samples without saying what small means, which is the kind of sentence a reader cannot act on. Here is the number. A point-biserial is a correlation, so its 95 per cent interval can be put around it with Fisher's transformation, and the interval is wide for a long time. At 30 responses, a point-biserial of 0.20 has a 95 per cent interval running from about −0.17 to 0.52. The review threshold in the paragraph above cannot be applied at that sample size at all, because the number cannot tell a broken item from a good one. At 50 responses the interval is −0.08 to 0.45. It takes about **97 responses** before a point-biserial of 0.20 is distinguishable from zero, and about **340** before it is distinguishable from 0.30 — that is, before the threshold can actually sort one item against another. Difficulty behaves better but not well. A p of 0.52 is a proportion, so its standard error is the square root of p(1 − p) ÷ n: at 30 responses the 95 per cent interval is 0.34 to 0.70, at 100 it is 0.42 to 0.62, and at 500 it is 0.48 to 0.56. A p reported to two decimals on 30 sittings implies a precision of one point in a hundred where the honest figure is one in three. Run the worked example below through those intervals and it stops reading like a verdict. On 30 sittings, item 12's healthy 0.41 is really 0.06 to 0.67, and item 14's −0.18 — the one flagged as a miskey — is really −0.51 to 0.19, an interval that includes zero. The item may well be miskeyed, and a human reading the stem is the check that would establish it; the statistic on 30 responses is not. One published guideline is worth reading with its own qualification attached. Cappelleri, Lundy and Hays, writing on classical test theory for measure development, say that for classical item analysis "starting with a sample of 30 to 50 subjects may be reasonable" — and in the same passage note that sample-size adequacy at that early stage "does not have the same level of importance as it would in later (and especially confirmatory) phases". The 30 to 50 figure is for exploratory work on a measure still being built. It is routinely quoted without the second half, and the second half is the half that stops it being used to retire items in a live bank. Two practical rules follow. Use small-sample item statistics to generate questions for a human reviewer, never to make the retirement decision on their own. And when a vendor or an internal report hands you a table of item statistics, ask for n before you read a single coefficient — an item-analysis report with no sample size printed on it is a table of numbers with unknown width, and those are published more often than not. The caveat on the arithmetic, since this entry is about not over-reading numbers: Fisher's transformation assumes both variables are continuous, and a point-biserial has a dichotomous half. The intervals above are close approximations and, if anything, slightly optimistic. Related reading: [reliability](https://www.assessall.com/guides/glossary/r/reliability) for what these statistics feed, and [differential item functioning](https://www.assessall.com/guides/glossary/d/differential-item-functioning) for the separate question of whether an item behaves differently across groups. --- Source: https://www.assessall.com/guides/glossary/i/item-analysis