Pharmacovigilance and Adverse Event Case Handling Assessment for Drug Safety and Clinical Operations TeamsThe rash cleared when the tablet was stopped. Did you call the case right, and did you know you had?

Methodology: Forty original exercises across six formats: twelve single-choice case calls, eight confidence probes on a slider, seven true-or-false claims, five select-every-one-that-applies lists, four match-the-following exercises and four ordering exercises. One response instruction is declared for the whole instrument and it is a KNOWLEDGE instruction: what the case supports under the procedure stated in the exercise, never what the respondent would do or feel. The eight confidence probes are part of the same knowledge instruction: each names a case the respondent has just called and asks for a percentage chance that their own earlier call was right (four probes) or wrong (four probes), which is a metacognitive judgement about their own answer and is never scored or described as a behavioural tendency, a trait or a personality style. Construct statement: this measures whether a person handling an adverse event case reaches the right call under a stated procedure, across seriousness and the reporting clock, causality reasoning, case completeness and follow-up, AND whether they know how sure to be of it; it does not measure knowledge of any one country's regulations, any regulator's forms or timelines, any terminology dictionary, any safety database product, clinical judgement about a real patient, medical writing, speed, or how a person works under a case backlog, and it is not a qualification. Scoring is C9, calibration scoring. ACCURACY is computed over the thirty-two keyed exercises: each answered exercise yields a quality in the unit interval and a chance term from its declared prior, which is the empirical marginal the instrument declares for that exercise: option_base_rate on single-choice and true-or-false items (the declared share of working case handlers expected to choose each option), per-option select rates on select-all items combined through the expected F1, pair_base_rate on matching items and position_base_rate on ordering items. The chance-corrected score is (raw minus expected chance) over (one hundred minus expected chance), so zero is the ordinary case handler answering at the declared marginals, never half marks; the marginals are printed on the page with their spread, and observed data will replace them. An unanswered exercise leaves the numerator, the denominator and the chance term together rather than scoring zero. An empty sitting scores exactly zero. Fewer than twenty keyed answers of thirty-two is refused, not reported: the refusal is printed where the dumbbell would have been, and no family, no calibration figure and no placement is printed beneath it. CALIBRATION is computed over the eight probes. The stated chance p is the slider value over one hundred on a probe phrased as the chance of being right, and one minus that on a probe phrased as the chance of being wrong; the outcome o is one if the named call was right. The Brier score B is the mean squared difference between p and o, decomposed after Murphy into reliability (the mean squared gap between stated and observed within four declared confidence bins, smaller is better), resolution (how far the hit rate within each bin departs from the overall hit rate, larger is better) and uncertainty (a property of the calls, not the respondent). Reliability and resolution are reported SEPARATELY and never summed, because a person can be well calibrated and undiscriminating, or discriminating and over-confident. The signed gap between mean stated chance and hit rate is the direction: over-sure or under-sure. Four probes attach to calls declared hard (keyed prior at or under .40) and four to calls declared easy (at or over .55), so a respondent who states the same high number on every probe is scored on calls they are likely to get wrong as well as right; the builder counts the split it received and prints it, and a respondent whose raw slider values do not move between right-phrased and wrong-phrased probes is flagged for review rather than scored as calibrated. Eight probes is the floor for a calibration figure, and the page says so: with all eight answered and their calls answered, the Brier score, reliability, resolution and direction are printed each with a standard error from a deterministic bootstrap and a 68 and 95 per cent band; with four to seven, only a placement in words is printed for the direction and the figures are refused; with fewer than four, the calibration strand is refused entirely. There is no zero on a calibration scale, so three reference respondents are priced on the reader's own right-and-wrong pattern instead: fifty on every probe, ninety on every probe, and the declared prior of each call stated as the confidence. The report is D4, the dumbbell of difference, adapted to self against self: one row per case family, a filled square for the share of that family's keyed exercises the respondent got right and a hollow circle for the mean stated chance on that family's two probes, each carrying a word as well as a shape, joined by a line and sorted by the size of the gap, largest first. The accuracy dot carries the declared expected-chance tick for that family and a binomial standard-error band; the confidence dot carries a band from a declared standard deviation of stated confidence of eighteen points, an assumption observed data will replace. A gap is printed in words and points only where the two 68 per cent bands do not cross; where they cross, the row prints not distinguishable at this length instead of a gap, and two probes per family is stated on the page as the reason that will often be so. Subscore discipline: each family carries eight keyed exercises; at the assumed inter-item correlation of .24 a family carries a chance-corrected figure only when all eight are answered and its omega, printed unrounded, is at or above .70; otherwise a three-way placement in words, never a number and never a percentile. The composite over thirty-two exercises has an assumed omega of about .91 and carries a figure with its 68 and 95 per cent bands from SEM equal to SD times the square root of one minus omega, with SD declared at twenty-four on the corrected scale from a modelled mixed-ability reference population run through this builder. Reliability weighting is used only when the omega spread across families exceeds .15; otherwise unit weights, stated. Every omega is McDonald's omega estimated from item count and the assumed inter-item correlation because no live data exist yet, and the page says so. Fixed habits are priced through the real builder before shipping: always serious, never serious, always the reporter's view, ninety on every probe and fifty on every probe each land at or below the ordinary case handler on the strand they touch. Two true-or-false claims tagged as a pair probe the same rule, that a sex and an age group identify a patient, through different cover stories, and disagreement between them is printed for review rather than scored. A careless responding composite is printed to whoever reviews the sitting as a flag count, never to the respondent as a judgement. Sources drawn on for the constructs and the scoring: Brier (1950) on the verification of probability forecasts; Murphy (1973) on the reliability-resolution-uncertainty decomposition; Lichtenstein, Fischhoff and Phillips (1982) on the calibration of probabilities; Yates (1982) on external correspondence in probability judgement; Koriat, Lichtenstein and Fischhoff (1980) on the reasons for confidence; Moore and Healy (2008) on the three forms of overconfidence; Hill (1965) on association and causation; Naranjo and colleagues (1981) on a structured probability scale for adverse drug reactions; Meyboom and colleagues (1997) on causality assessment in pharmacovigilance; Agbabiaka, Savovic and Ernst (2008) on the systematic review of causality assessment methods; Edwards and Aronson (2000) on the definition and diagnosis of adverse drug reactions; Waller and Evans (2003) on the conduct of pharmacovigilance; Haladyna, Downing and Rodriguez (2002) on item writing and cue control; and the three public frameworks named in framework_note, each cited once and with affiliation disclaimed. All forty exercises are original works written for this instrument. No case describes a real patient, reporter, product, company or regulator. No commercial instrument's items or name are used or implied, no licensed terminology dictionary is named, no safety database product is named, and no competitor is named. This is not a regulatory qualification, it does not authorise anybody to submit a safety report, no real case is assessed, and nothing in it is medical advice.