--- title: "77% of India's Web Traffic Is Mobile. Five Myths About Assessing Candidates on Phones." description: "Across 3.5 million applicants, mobile GMA scores ran substantially lower - and in a five-assessment study, measurement invariance held while mobile SJT means sat half a standard deviation below desktop. Five myths about device effects, tested against the evidence." canonical: https://www.assessall.com/blog/77-of-indias-web-traffic-is-mobile-five-myths-about-assessing-candidates-on published: 2026-10-17 updated: 2026-10-17 author: "AssessAll Editorial" source: AssessAll --- # 77% of India's Web Traffic Is Mobile. Five Myths About Assessing Candidates on Phones. Across 3.5 million applicants, mobile GMA scores ran substantially lower - and in a five-assessment study, measurement invariance held while mobile SJT means sat half a standard deviation below desktop. Five myths about device effects, tested against the evidence. A device effect is a difference in assessment scores caused by the hardware a candidate tested on rather than by the ability being measured. It is construct-irrelevant variance: the same person, the same items, a different number. In selection testing the effect is largest on speeded tasks and close to zero on untimed ones. That matters more every year, because for a growing share of candidates the device is not a choice. Mobile accounted for 77.3% of web traffic in India in September 2026 against 22.1% for desktop, [according to StatCounter](https://gs.statcounter.com/platform-market-share/desktop-mobile-tablet/india). The GSMA's *State of Mobile Internet Connectivity 2026*, [published on 15 September 2026](https://www.gsma.com/newsroom/press-release/gsma-warns-of-emerging-global-ai-divide-as-three-billion-people-remain-offline-and-smartphone-component-costs-soar/), counts 4.8 billion mobile internet users and 3.4 billion people who do not use mobile internet at all even though more than 90% of them live under coverage. It also reports that at the end of 2025 an entry-level internet-enabled handset cost the poorest 20% in low- and middle-income countries 44% of average monthly income, rising to 76% in Sub-Saharan Africa. A pipeline drawn from those markets is phone-shaped, whatever your test was designed on. Here are five things practitioners commonly believe about that, each tested against the published evidence. ## Myth 1: The research settled it — mobile and desktop scores are equivalent The literature is split, and the split is informative. Arthur and colleagues [analysed 3,575,207 job applicants](https://doi.org/10.1111/ijsa.12062) who completed a high-stakes unproctored internet-based assessment between January 2011 and April 2012. Personality scores were similar across device types; general mental ability scores were "substantially lower" on mobile devices. Brown and Grossenbacher, [testing GMA across device groups](https://doi.org/10.1111/ijsa.12160), found no meaningful differences and only a marginal screen-size effect. The contradiction has an old explanation. Mead and Drasgow's [1993 meta-analysis in *Psychological Bulletin*](https://doi.org/10.1037/0033-2909.114.3.449) pooled 159 correlations — 123 from timed power tests, 36 from speeded tests — and estimated the cross-mode correlation at .97 for power tests but only .72 for speeded ones. Speededness, not mode, was doing the work. Thirty-three years later the same mechanism is visible in hardware. Passell and colleagues tested [59,587 visitors to TestMyBrain](https://link.springer.com/article/10.3758/s13428-021-01597-3) and found simple reaction time slower on Android (β = 0.56 relative to Windows, with age, gender and education controlled) and on iPhone and iPad (β ≈ 0.21–0.23). Comparing a touchscreen iPad against a mouse-and-keyboard Mac isolated input method rather than screen size (β = −0.36), and untimed vocabulary showed no comparable mobile slowdown. The useful question is therefore not whether candidates test on phones. It is how much of your score is milliseconds. ## Myth 2: Measurement invariance proves the scores are comparable This is the most expensive error of the five, because the evidence that reassures people is the evidence that does not answer the question. Morelli and colleagues [tested invariance across five selection assessments](https://doi.org/10.1111/ijsa.12063) — a cognitive-ability-type measure, two biodata instruments, a multimedia work simulation and a text-based situational judgment test. Invariance held for all of them. Yet mobile users' latent means were **half a standard deviation lower** than non-mobile users' on the SJT. Arthur et al. reported the same combination: invariance supported, GMA means substantially lower. Invariance asks whether the instrument measures the same construct in the same way for both groups. It does not ask whether the two groups obtain the same scores, and a cut score is applied to scores, not to factor loadings. The [ITC and ATP *Guidelines for Technology-Based Assessment* (2022)](https://www.intestcom.org/page/28) put this plainly. Guideline 7.4 places the burden of evidence on anyone claiming score comparability "across different technologies, devices, and administrative conditions". Guideline 7.6 requires that psychometric evidence for comparability include "an evaluation of the properties and differences in the shape of the score distribution, reliability, and standard error of measurement". An invariance test is one line in that file, not the file. ## Myth 3: Responsive design fixes it Responsive rendering is necessary. The same guidelines ask that the interface respond to the form factor, orientation and available screen area of the device, and that native interactions such as pinch-zoom be supported. But a layout is not a latency. If part of a candidate's score is how fast a tap registers, re-flowing the items changes nothing that matters. Nor is a noisy bedroom the explanation. Traylor and colleagues [randomly assigned 425 participants](https://doi.org/10.1111/ijsa.12315) across combinations of testing environment and device; those in the distracting environment reported significantly more distractions and scored no differently, a result that replicated within the study. The authors concluded that self-selection and test stakes remain the hypotheses worth testing. Which is to say: the device effect observed in operational hiring data may be partly a statement about *who* chooses a phone, not only about the phone. ## Myth 4: It is a rounding error — hardly anyone tests on a phone In Arthur's 2011–12 data only 1.93% of applicants used a mobile device — though that was still more than 69,000 people. No operational dataset of comparable size has been published since, which is itself the finding: most employers do not know their own number, because device type is not in the score record. Device choice is not random. Arthur et al. found mobile use slightly more likely among women, African-American and Hispanic applicants, and younger applicants. ACT's 2018 survey of 7,233 US high school students found: - **Students with access to only one device at home:** 14% of respondents, of whom 56% said that device was a smartphone, and 85% were classified as underserved on income, parental education or race/ethnicity. - **Home internet quality in that group:** 15% described it as unpredictable and 3% as terrible, against 10% and 1% among students with two or more devices ([ACT Research, 2018](https://www.act.org/content/dam/act/unsecured/documents/R1698-digital-divide-2018-08.pdf)). When device correlates with group membership and device depresses scores, the device is a pathway to adverse impact — which under [the EEOC's guidance on employment tests](https://www.eeoc.gov/laws/guidance/employment-tests-and-selection-procedures) is something you are expected to detect and justify, not discover in litigation. ## Myth 5: Blocking phones is the fair, safe default It is sometimes defensible. The ITC/ATP guidelines explicitly allow restricting classes of devices, form factors, input types and operating systems in order to offer comparable testing conditions to all candidates. But the cost of that restriction lands on the candidates least able to absorb it: the mobile-only applicant in a market where a basic handset costs most of a month's income. Blocking converts a measurement problem into an access problem, and an access problem is an adverse-impact problem with tidier paperwork. ## What to do instead 1. **Record the delivery context.** Device type, operating system, input method and connection quality, stored with the score. You cannot audit a variable you do not capture. 2. **Split your own distribution.** Compare means, standard deviations, pass rates and standard errors by device, as Guideline 7.6 asks — not just a global invariance test. 3. **Audit speededness before anything else.** Mead and Drasgow's .72 is the single most actionable number here. If the construct does not require speed, remove the clock or set a generous limit, and the device effect largely goes with it. 4. **Prefer constructs that survive the medium.** Scenario-based and work-sample formats scored on the content of a response travel better across hardware than speeded clerical drills. AssessAll's AI-graded scenarios and proctoring integrity bands are built for unsupervised remote sittings of exactly this kind. 5. **Do not adjust scores by device.** Removing the cause is defensible; a device-based score correction is a new fairness claim you would then have to defend. 6. **If you must restrict, restrict by capability and say so early.** Publish the technical requirements before candidates invest an hour, and offer an alternative sitting to anyone the restriction excludes. 7. **Read pass rates by group and device together.** Either variable alone can look clean while the interaction does not. Device effects are not a reason to avoid mobile assessment; they are a reason to measure your own. The evidence says the risk concentrates in speeded content and that the usual reassurance — a passing invariance test — is the one piece of evidence that cannot rule it out. --- Source: https://www.assessall.com/blog/77-of-indias-web-traffic-is-mobile-five-myths-about-assessing-candidates-on