Training Evaluation and Programme Effectiveness Assessment for L&D and HR TeamsA programme defended on how much people enjoyed it has not been defended. Do you know what would defend it?

Methodology: Thirty-eight original exercises across six formats: ten single-choice judgement items, eight allocation sliders (two programme briefs, each split across the four levels of evaluation), six select-every-that-applies exercises, six binary claims, four matching exercises and four ordering exercises. One response instruction is declared for the whole instrument and it is a KNOWLEDGE instruction: what the published evidence supports for the programme described, never what the respondent would personally do or what is usual in their organisation. Construct statement: this measures applied judgement about how to evaluate a training programme, meaning which level of evidence a decision needs, what comparison a claim of effect rests on, what the work environment does to transfer, how to read an evaluation figure without over-claiming, and what an honest report to a sponsor says; it does not measure the respondent's ability to run an evaluation, their statistical skill, their programme-design ability, whether their own programmes work, or their standing with any sponsor, and it is not a certification in evaluation. The four levels of training evaluation are treated throughout as a public structure and are named by their descriptions, reaction, learning, behaviour on the job and organisational result; no evaluation product, model name or author's name is used anywhere in the instrument, and no affiliation with any such product is claimed or implied. Scoring has two strands that are never combined. The KEYED strand is the thirty keyed exercises: each is scored on the platform's own rule for its type, expressed as a quality from zero to one, and chance-corrected against a declared answer prior authored on every option, claim, matching pair and ordering position, so that zero means a respondent who answers the way ordinary practice answers rather than a uniform guess; the strand carries the headline number with its standard error band, its reliability stated as McDonald's omega estimated from item count and a declared inter-item correlation of .20 until live data replace it, and a three-way placement rather than a figure for each of the five competencies because six items cannot carry a number. The ALLOCATION strand is an expert-vector allocation match: the respondent's eight shares are correlated with a reference allocation, so that the level of effort drops out and only the shape is scored, and the correlation is corrected against a Monte-Carlo null drawn from the declared answer distribution on each slider, in which the reaction-heavy allocation of ordinary practice carries the mass. The reference allocation is an AUTHORED CONSENSUS read off the published evaluation literature. No rater panel has sat, no inter-rater agreement has been computed, and an expert vector without five or more raters and a reported intraclass correlation is one reading of the literature rather than a measured standard; for that reason the allocation strand is refused the match figure that a measured agreement would have licensed, and the report prints a three-way placement, both vectors in full, and the reason, never a percentile and never a precise match score. The two errors the instrument is keyed against are the two that evaluation practice most often makes: measuring only reaction, and claiming an organisational result from a design that cannot support the claim. Both are errors of commission; the distractors also carry errors of omission, such as deferring every finding to a longer follow-up, skipping the learning measure, or leaving the sponsor's question unanswered. Keying sources: Alliger and Janak (1989) on the assumptions of the four-level structure; Alliger, Tannenbaum, Bennett, Traver and Shotland (1997), the meta-analysis of relations among training criteria; Sitzmann, Brown, Casper, Ely and Zimmerman (2008) on reactions and learning; Holton (1996) on the flawed four-level model; Bates (2004) on the limits of evaluation practice; Kraiger, Ford and Salas (1993) on cognitive, skill-based and affective learning outcomes; Baldwin and Ford (1988) and Blume, Ford, Baldwin and Huang (2010) on transfer of training; Burke and Hutchins (2007) on transfer; Ford, Quinones, Sego and Sorra (1992) on opportunity to perform; Gollwitzer and Sheeran (2006) on implementation intentions; Arthur, Bennett, Edens and Bell (2003) on training effectiveness and effect sizes; Salas, Tannenbaum, Kraiger and Smith-Jentsch (2012) on the science of training; Shadish, Cook and Campbell (2002) on experimental and quasi-experimental designs; Sackett and Mullen (1993) on evaluation beyond formal experimental design; Cohen (1988) on effect-size conventions; Rossi, Lipsey and Freeman (2004) on evaluation practice; and the recurring practitioner surveys of evaluation practice, which report reaction measured on nearly every programme and organisational results on a small minority, and which are the source of the reaction-heavy answer prior. Every exercise is an original work written for this instrument; none reproduces an item from any commercial instrument, and no affiliation with any evaluation product, framework owner or certification body is claimed.