What is Test–retest reliability?

Also called Temporal stability

Test–retest reliability is the correlation between the scores of the same people measured twice with an interval between sittings. It is the most intuitive form of reliability and the least forgiving, because it exposes both measurement noise and genuine change in the person, and the two are difficult to separate.

Why the interval has to be reported

A retest after two days is inflated by memory of the previous answers. A retest after two years is deflated by real development in the person. Neither is wrong, but a coefficient without its interval cannot be interpreted, and the interval is often omitted from marketing material precisely because the short ones look better.

Ask for the interval, the sample size, and the population. A stability figure from a student sample over three weeks says little about working adults over a year.

What the interval is worth, in numbers

The largest quantitative answer to this question is Roberts and DelVecchio’s 2000 review in Psychological Bulletin, which compiled 3,217 test–retest correlation coefficients from 152 longitudinal studies of personality traits. Holding the interval constant at 6.7 years, rank-order consistency rose with the age of the sample: .31 in childhood, .54 across the college years, .64 at around age 30, and .74 between 50 and 70, where it plateaued.

Two things follow for anyone reading a vendor’s stability figure. The population is not a footnote — the same instrument is measurably more stable on a mid-career sample than on a graduate one, so a coefficient collected on students understates what it will do in a management population and a coefficient collected on fifty-year-olds overstates it.

And the interval is not a footnote either. The same review found the longitudinal interval negatively related to consistency independently of age, and estimated average trait consistency at about .55 over one year falling to roughly .25 over forty. A figure quoted without its interval is not a conservative claim; it is an uninterpretable one.

The type-flip problem

Instruments that convert a continuous score into a category — a type, a letter, a colour — have a reliability problem their continuous scores do not. If a scale is cut at its midpoint, someone who scores near that midpoint will flip categories on a retest even when their underlying score barely moves.

This is why a questionnaire can report respectable continuous stability and still hand a substantial share of people a different type a few weeks later. Both figures can be true at once, and only the second is what the person experiences.

What a stability coefficient does to the confidence band

The reason to insist on the retest figure rather than the internal-consistency one is that they produce different error bars on the same score, and the difference is large enough to change a decision. The standard error of measurement is the scale's standard deviation times the square root of one minus the reliability coefficient — SEM = Sx√(1 − rxx′) in Harvill's NCME module — and whichever coefficient you put in is the question you are answering.

Put a typical published alpha of 0.85 into it, on a scale with a standard deviation of 10 points, and the SEM is 3.9 points: a 95% confidence band of about ±7.6, or 15 points wide. Put in 0.55, the average one-year trait consistency from the review above, and the SEM is 6.7 points: a band of ±13.1, or 26 points wide. Same instrument, same candidate, a band 1.73 times wider — and the second one is the honest band if the decision the score informs plays out over a year rather than inside the sitting.

The two coefficients are not rivals and neither is wrong. Internal consistency asks whether the items measured one thing this morning. Stability asks whether the result would still be there next spring. A hiring decision is made this morning and lived with for years, so the band that matters is usually the wider one, and it is the one almost never printed.

One caution, because conflating these is the error this entry exists to prevent. A rank-order consistency coefficient collected over years is not the retest reliability of an instrument: it bundles measurement error together with genuine development in the person, and no arithmetic separates them. Using it in the SEM formula does not yield the instrument's error — it yields the width of the band you should treat a score as carrying if you want the score to stand up over that horizon. That is a different and, for selection, usually more useful quantity.

Solve the formula the other way and one threshold falls out. A 95% band of ±1.96 SEM fits inside one whole standard deviation of the scale only when the reliability coefficient is about 0.935 or better. Below that, the band around a single reported score is wider than the spread of the whole population on that scale. Note that the banding entry on this site solves a related question — the band around the DIFFERENCE between two candidates' scores — and gets a different threshold, because a difference carries an extra factor of √2. Two questions, two widths, and a report that prints neither is inviting the reader to assume there is no width at all.

From a reliability coefficient to the band around one score

SEM = Sx × √(1 − rxx′), where Sx is the scale's standard deviation
95% confidence band around a reported score = ±1.96 × SEM
On a scale with SD = 10 —
Internal consistency 0.85 → SEM 3.9 → ±7.6 points (band 15 points wide)
One-year trait consistency 0.55 → SEM 6.7 → ±13.1 points (band 26 points wide)
College-years stability 0.54 → SEM 6.8 → ±13.3 points
Age 50–70 stability 0.74 → SEM 5.1 → ±10.0 points
Forty-year interval 0.25 → SEM 8.7 → ±17.0 points (band 34 points, wider than three SDs)
Band equals one whole SD of the scale at rxx′ ≈ 0.935

Run the first two lines on whatever coefficient the manual reports, then run them again on the retest figure if one exists. If the vendor publishes only an alpha, the band you have computed is the narrowest of the two available and you should say so out loud before anybody is ranked by it.

Not the same as internal consistency

Internal consistency is measured inside one sitting and describes whether the items hang together. Test–retest is measured across two sittings and describes whether the result holds. A scale can score well on the first and poorly on the second.

Cronbach's alpha

Sources

Check a selection process against the four-fifths rule

Free, no signup, computed in your browser — with the remedy, not just the verdict.

Open the calculator

Last reviewed 2026-09-29