What is Utility analysis?
Also called Selection utility, Brogden–Cronbach–Gleser model
Utility analysis estimates the money a selection procedure adds, rather than its correlation with performance. The standard model multiplies the validity coefficient by the standard deviation of employee output value, by the average standardised score of the people hired and by how long they stay, then subtracts the cost of assessing every applicant.
The equation, and what each term is really doing
The gain is the number hired, times tenure in years, times the validity coefficient, times SDy — the standard deviation of the value of employee output in money — times the mean standardised test score of those hired, less the total cost of testing. Brogden set it out in 1949, Cronbach and Gleser generalised it in 1965, and Schmidt, Hunter, McKenzie and Muldrow put it in front of practitioners in 1979.
The mean standardised score of those hired is set entirely by the selection ratio: 1.755 standard deviations at one hire in ten, 0.798 at one in two, 0.109 at nineteen in twenty. That is how the selection ratio enters the money model, and it is why the same procedure has a different financial value at two employers with identical validity evidence.
SDy is the weak term and everyone in the field knows it. Forty per cent of salary is the conventional lower-bound estimate. Different estimation methods applied to the same job routinely disagree by a factor of two, and because SDy multiplies the whole expression, that disagreement passes straight through to the answer.
The output is not believable, and the field has the experiment to prove it
Work a realistic case. Five hundred applicants, fifty hires, a validity of 0.31, a salary of 60,000, SDy at 40% of that, two years of expected tenure and 25 per applicant to assess. The model returns roughly 1.29 million net. It also says each hire is worth an extra 13,000 a year — about 22% of their salary — and that the break-even validity is 0.003, which means any procedure correlated with performance at all pays for itself many times over.
A model that says every test is a bargain is not giving you information. That should read as a warning about the model rather than as a selling point for the purchase.
Latham and Whyte tested the persuasion directly in 1994. They gave 143 experienced managers either a validity argument for a selection procedure or the same argument plus a utility analysis showing substantial net financial benefit. The utility analysis reduced their support for adopting it. Whyte and Latham repeated the study in 1997 with an internationally recognised expert presenting the analysis, on video and then live, and the effect held. Cronshaw — the expert in question — published a reply in the same issue. This is a rare case of a field running an experiment on its own persuasion technique and finding it counterproductive.
What it is still genuinely good for
Comparisons rather than absolutes. Run two candidate procedures under the same SDy assumption and most of the error cancels: the ranking survives even though the currency figure does not. That is the defensible use, and it is the one that answers the question a buyer actually has, which is A or B rather than how many millions.
Break-even questions, too, because they are asked in the direction the model handles best. What validity would this procedure need in order to break even at our cost per applicant? If the answer is far below anything any instrument plausibly delivers, then the purchase decision is not really about validity — it is about throughput, candidate experience or adverse impact, and it should be argued on those terms instead of dressed up in money.
And always present the increment over what you already do, never the gross. A gain your existing structured interview is already capturing is not a gain the new purchase creates.
Brogden–Cronbach–Gleser, one realistic case
- 500 applicants, 50 hires → selection ratio 0.10, mean standardised score of those hired 1.755
- Validity 0.31 × SDy 24,000 × 1.755 = 13,057 per hire per year
- × 50 hires × 2 years tenure = 1,305,720 gross
- − (25 per applicant × 500 applicants) = 12,500 testing cost
- Net estimated gain 1,293,220 — and the break-even validity is 0.003
All figures are in one unnamed currency — the arithmetic is identical in any of them. SDy is taken as 40% of a 60,000 salary, the conventional lower bound. Read the last line first: a model whose break-even is 0.003 cannot distinguish a good procedure from a barely-valid one, which is the honest limit of the method.
Not the same as return on investment
ROI, as a finance team uses it, compares cash that actually moved. Utility analysis is a forecast resting on a psychometric estimate, an economic estimate and an assumption about tenure — none of the three observed. The two are quoted in the same units and have entirely different epistemic status. Label a utility figure an estimate with a range, or it will be read as an accounting number and, per Latham and Whyte, disbelieved.
Sources
- Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2(2), 171–183 — the linear utility model
- Schmidt, F. L., Hunter, J. E., McKenzie, R. C. & Muldrow, T. W. (1979). Impact of valid selection procedures on work-force productivity. Journal of Applied Psychology, 64(6), 609–626
- Latham, G. P. & Whyte, G. (1994). The futility of utility analysis. Personnel Psychology, 47(1), 31–46
- Whyte, G. & Latham, G. (1997). The futility of utility analysis revisited: when even an expert fails. Personnel Psychology, 50(3), 601–610
Read next
Related terms
Check a selection process against the four-fifths rule
Free, no signup, computed in your browser — with the remedy, not just the verdict.
Last reviewed