---
title: "95% of Referees Follow Through. 36% Mention a Weakness. Six Myths About Reference Checks."
description: "The 2022 meta-analytic revision of selection validity set reference checks aside for insufficient information. In MSPB's account of a 905-provider study, 95% of referees followed through, 64% discussed rehire and 36% discussed weaknesses. Six myths, tested against evidence."
canonical: https://www.assessall.com/blog/95-of-referees-follow-through-36-mention-a-weakness-six-myths-about-reference
published: 2026-11-12
updated: 2026-11-12
author: "AssessAll Editorial"
source: AssessAll
---

# 95% of Referees Follow Through. 36% Mention a Weakness. Six Myths About Reference Checks.

The 2022 meta-analytic revision of selection validity set reference checks aside for insufficient information. In MSPB's account of a 905-provider study, 95% of referees followed through, 64% discussed rehire and 36% discussed weaknesses. Six myths, tested against evidence.

A reference check is a structured enquiry to a named former manager or colleague about how a candidate actually worked. Its most-quoted validity figure, .26, comes from research assembled before 1998. When a 2022 meta-analytic revision rebuilt selection validity, reference checks were set aside for insufficient information rather than re-estimated. The practice is near-universal; the evidence behind it is not.

That gap matters because the reference call usually sits at the end of the funnel, after the assessment and the interview, where it carries veto power over decisions everything else was designed to make. Six beliefs hold that arrangement in place. Here is what the published evidence says about each.

## Myth 1: The reference check predicts performance

The figure in circulation, .26, traces to the pre-1998 literature summarised in the [Schmidt and Hunter validity table](https://www.assessall.com/guides/evidence/schmidt-hunter-1998-validity-table). Sackett, Zhang, Berry and Lievens rebuilt that table in the *Journal of Applied Psychology* in 2022, correcting the range-restriction adjustments that had inflated several entries ([record](https://ink.library.smu.edu.sg/lkcsb_research/6894/)). Reference checks do not appear among their updated estimates. They appear in the paper's short list of procedures excluded for insufficient information, still carrying the old .26, with no new coefficient, no source meta-analysis and no study count.

Set that against the predictors the same paper did re-estimate:

- **Re-estimated and still strong**
    - Structured employment interviews: .42 (was .51)
    - Job knowledge tests: .40 (was .48)
    - Empirically keyed biodata: .38 (was .35)
- **Re-estimated and substantially lower**
    - Work sample tests: .33 (was .54)
    - Cognitive ability tests: .31 (was .51)
    - Integrity tests: .31 (was .41)
    - Unstructured interviews: .19 (was .38)
    - Years of job experience: .07 (was .18)

A procedure with no current estimate is not a procedure with a low estimate — it is one nobody has recently measured. Treating .26 as live evidence is a claim the literature does not support.

## Myth 2: Referees will tell you the problems

They answer the call. They just do not answer that question. The US Merit Systems Protection Board summarised a 2019 study in the *International Journal of Selection and Assessment* that contacted 905 supervisors and coworkers who had been asked for a reference in the previous year ([Issues of Merit, January 2023](https://www.mspb.gov/studies/newsletters/Issues_of_Merit_January_2023_1992828.pdf); the study itself is [Hedricks et al., 2019](https://doi.org/10.1111/ijsa.12243)). In MSPB's account, 95% of reference providers agreed to serve and followed through — and a separate study of 150,000 candidates described in the same newsletter reported 85.3% compliance.

What they discussed, as a share of providers:

- Type of work the candidate did: 81%
- The candidate's performance: 78%
- Job-related strengths: 71%
- Whether they would rehire the candidate: 64%
- Potential for future success: 60%
- Weaknesses: 36%

Among those who declined to serve, 52% cited reluctance to report negative information. Asked why they agreed, 43% said to reward a competent worker and 22% to help with a career change. None reported a motivation to hold a poor candidate accountable.

The same ceiling shows up wherever reference ratings are studied systematically. A review of 90 standardised letters of recommendation from 82 urology residency applicants found applicants clustered in the highest tiers on five of six rated domains in one letter format and all nine in the other; the authors concluded there is "a lack of variability among domain ratings" that "makes it hard to differentiate between highly qualified" applicants ([Urology, 2022](https://www.goldjournal.net/article/S0090-4295%2821%2900823-2/fulltext)).

A measure on which nearly everyone scores at the top discriminates between nobody. That is a range problem, not a candour problem, and it does not improve by asking more warmly.

## Myth 3: Silence from a referee means nothing

This is the one myth the evidence pushes back on in the useful direction. MSPB reports that earlier work found "a reliable positive relationship between the number of uncontactable candidate references and future performance problems," citing [Hedricks, Robie and Oswald (2013)](https://doi.org/10.1111/ijsa.12020). No effect size is published in the newsletter, so treat this as a signal worth logging rather than a scored variable.

The operational implication is narrow but real: record how many named referees could not be reached, consistently, for every candidate. Unreachable is data. It is also the only part of a reference check that is hard to inflate.

## Myth 4: A background check and a reference check do the same job

They fail in opposite directions. Background verification confirms facts — employment dates, job titles, qualifications, records — and its failure mode is a false record. A reference check solicits judgement, and its failure mode is a true but uninformative one. Collapsing them into one "BGV stage" means neither gets designed.

The legal treatment differs too. In the United States, a report bought from a third-party screening firm is a consumer report under the Fair Credit Reporting Act: it requires a permissible purpose and the candidate's written authorisation, with notice before adverse action ([15 U.S.C. §1681b](https://www.law.cornell.edu/uscode/text/15/1681b)). Blanket exclusions based on criminal records carry Title VII disparate-impact exposure under the EEOC's 2012 enforcement guidance ([PDF](https://www.eeoc.gov/sites/default/files/migrated_files/laws/guidance/arrest_conviction.pdf)). India has no FCRA analogue, but assessment and verification data on a candidate is personal data under the DPDP framework, with consent and erasure duties attached — [covered here](https://www.assessall.com/blog/your-assessment-scores-are-personal-data-what-indias-dpdp-rules-require-before).

## Myth 5: A reference check is legally safer than a test

Not in the United States, where the Uniform Guidelines define a selection procedure as "any measure, combination of measures, or procedure used as a basis for any employment decision," explicitly reaching "informal or casual interviews and unscored application forms" ([29 CFR 1607.16(Q)](https://www.ecfr.gov/current/title-29/section-1607.16)). If a reference call changes who gets hired, it is a selection procedure, and the job-relatedness expectation applies. An undocumented phone call is not exempt from scrutiny; it is merely harder to defend, because nothing was written down.

## Myth 6: More referees fixes it

Volume does not fix a range problem. Structure does. The one reference procedure with published validity evidence in a peer-reviewed selection journal is a structured telephone protocol ([Taylor, Pajo, Cheung and Stringfield, *Personnel Psychology*, 2004](https://doi.org/10.1111/j.1744-6570.2004.00006.x)), and the US Office of Personnel Management's [reference-checking guidance](https://www.opm.gov/policy-data-oversight/assessment-and-selection/other-assessment-methods/reference-checking/) likewise rests on structuring the call and contacting more than one source. The mechanics are the same ones that lift interviewer agreement from .44 to .74 when a panel scores a [structured guide](https://www.assessall.com/blog/separate-interviewers-agree-at-44-a-panel-at-74-how-to-build-a-structured):

1. Derive the dimensions from a job analysis, not from the hiring manager's last bad hire.
2. Ask every referee the same questions in the same order.
3. Use behaviourally anchored scales, and ask for comparative placement — how this person compared with others the referee has supervised in the role — rather than an absolute rating.
4. Fix the number of referees per candidate in advance.
5. Score independently, before discussion.
6. Record what was asked, who answered, who could not be reached, and what was said.

## Where the reference check still earns its place

For specific, checkable, late-stage facts, it remains the cheapest instrument available: did this person hold this role, in this scope, over these dates, and is there a disqualifying integrity event a verification search would miss. It is also appropriate where the alternative is nothing — a senior hire with no comparable assessment pathway, or a role where the critical behaviour only shows up over years and cannot be simulated. What it cannot do is substitute for measuring capability, because the people being asked are, by their own account, not there to tell you about weaknesses.

The design conclusion follows from that. Put the measurement load where it can be scored and audited — AssessAll's AI-graded scenario assessments and integrity bands run on pay-as-you-go credits at ₹30 / US$0.50 per assessment, which makes it practical to measure early rather than hope the reference call will catch what screening missed. Then let the reference check do the narrow job it is good at.

**Takeaway:** the reference check has no current validity estimate, a documented ceiling effect and a referee population that discusses weaknesses 36% of the time. Keep it for verification, structure it like any other selection procedure, and stop asking it to carry the hiring decision.

---

Source: https://www.assessall.com/blog/95-of-referees-follow-through-36-mention-a-weakness-six-myths-about-reference