Picture a warehouse operations manager in her early thirties, updating her resume on a Sunday night. She applies to four logistics companies in her metro area that week: a regional trucking firm, a food distributor, a fulfillment operator, and a parts warehouse that supplies an auto plant. The four compete for nothing except workers. Each application asks her to play a short set of online games, and by the end of the month all four have turned her down with a form letter.

Nothing she can see would tell her that the games were the same games, built by the same vendor, and that all four decisions may have been scored from one stored record of how she played, and the four hiring managers are no better placed to notice. For about four years that is roughly how screening worked at 156 employers that used a vendor called pymetrics, and a team of researchers from Stanford, Chapman University and Northeastern has now published what the whole picture looked like from above.1

One vendor, 156 employers

pymetrics supplied the researchers with its records: nearly 3.4 million applicants who submitted about 4.2 million applications between December 2018 and December 2022, to employers with a combined 225 billion dollars in annual revenue across eleven industries, among them finance, manufacturing and warehousing.1 Each employer got its own model. For every client role, pymetrics trained a separate classifier on the gameplay of at least 50 of that client's current employees, so on paper each company ran its own assessment. What the companies shared was the vendor and its games. A core set of twelve was the same across every position, and an applicant's results from them were stored and used again if the same person applied to another pymetrics job within 330 days.1

Four applications, one answer

Among applicants who applied to four positions, one in ten was not recommended for any of them. Among those who applied to ten, four percent were not recommended anywhere, and the researchers report that rate is higher than chance would produce if each decision had been made independently.1 The comparison they chose makes the point better than the raw figure does. They ran the same test on the largest earlier study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 companies, and there the share of people turned down everywhere was predicted almost exactly by employers deciding independently. The pymetrics data departed from that baseline.1

The paper is careful about one thing a reader should be careful about too. It cannot say whether the people screened out everywhere would have been good at the jobs, because nobody measured that. When the researchers ran every applicant through every model, though, each one would have been recommended by at least one of them, even though many real applicants were recommended for none of the positions they actually applied to.1

The pooled numbers passed

An earlier analysis by pymetrics' own researchers found no differences in selection rates that would draw scrutiny under U.S. employment discrimination law, and pooled across every position the new data agrees. Black applicants were selected at 0.839 of the rate of the most selected group and Asian applicants at 0.870, both above the four-fifths line federal guidelines use as the trigger for a closer look.1 U.S. guidelines judge discrimination one job at a time, though, so the researchers split the data into its 1,746 positions. At that level, 10.62 percent of positions showed adverse impact against Black applicants, and 25.87 percent of all applications from Black applicants went to those positions. For Asian applicants the share was 14.74 percent.1 The bar they set was strict, a ratio below four-fifths that also had to be statistically significant, with a correction for testing so many positions at once. Had those applicants been recommended at the rate of the most selected group, the paper estimates, another 11,513 applications from Black applicants and 29,320 from Asian applicants would have gone through.1

Every aggregate in this story is accurate. What an aggregate is allowed to average away is the whole problem.

What one employer can see

Two different problems sit in this paper, and a leader can act on only one of them. Adverse impact at a single position is something an employer can check with its own applicant data, provided it asks for results by job and does not settle for a vendor's pooled summary. The repeat rejection is harder. Single-employer studies, the authors write, "cannot observe whether rejection at Firm A predicts rejection at Firm B because they lack cross-employer applicant tracking."1 None of the 156 companies could have seen, from its own data, that the person it turned down had also been turned down by other companies scoring the same stored gameplay. The researchers addressed their recommendations to regulators, auditors and legislators. They ask for adverse impact to be measured per position, for closer surveillance of the vendor market, for agencies to monitor algorithmic monoculture directly, and for lawmakers to consider requiring researcher access to hiring algorithms.1

The authors spell out the study's limits. It covers one vendor, and they do not know how hiring managers used the recommendations once they had them. They also note that adverse impact alone does not make an employer's conduct illegal. And the data has some age on it, ending in December 2022, four months after pymetrics was acquired.1 What holds up is the mechanism. When many employers buy screening from one supplier, their decisions stop being independent of each other, and nothing in any single employer's reporting would show it.

I have written before about how fast a company will assert a causal story about AI that its own numbers cannot support. That piece was about layoffs, and it looked at how quickly a stated reason outruns the evidence behind it. Here the employers claimed nothing at all. The evidence that mattered was in a place none of them could look.

What a leader controls is smaller, and it is available now. Before signing, ask the vendor for adverse impact results broken out by position, for your own positions, and treat a pooled report as incomplete. Then ask how many other employers score candidates on the same assessment, and whether a candidate's results carry from one employer to the next. A large answer to either question belongs on the risk side of the ledger.

If your vendor told you tomorrow that the candidate you turned down last month had been turned down on the same stored results by every other client she applied to, what would you wish you had asked before you signed?