The findings raise new concerns for employers as AI increasingly becomes a fixture in recruiting. According to the World Economic Forum, about 90% of firms now use artificial intelligence to help vet job applicants.
Researchers at Stanford’s Institute for Human-Centered Artificial Intelligence analyzed 4 million job applications submitted to more than 150 employers using the same third-party hiring platform. They found that 26% of Black applicants and 15% of Asian applicants applied for positions where the AI system produced outcomes that met the Equal Employment Opportunity Commission’s standard for adverse impact. That means about 40,000 applications by these applicants could have advanced to the next stage of the hiring process.
The study’s findings mean that employers need to be mindful of potential discriminatory results from using AI during the hiring process. Courts have historically held employers liable for practices that create disparate impact on protected groups, and several recent lawsuits have challenged the use of AI in hiring under federal anti-discrimination laws.
Bias may be hidden
The study found that discriminatory outcomes can be difficult to spot because aggregate data often mask problems.
For example, an AI system might recommend Black applicants for one type of job but reject them for another. When all positions are combined, the disparities may disappear statistically even though significant differences exist for individual jobs.
That distinction matters because courts typically analyze disparate-impact claims on a position-by-position basis.
Perhaps the study’s most significant finding involves what researchers call “algorithmic monoculture.”
Many employers rely on the same small group of third-party vendors to screen candidates. As a result, the same algorithms may influence hiring decisions across hundreds of companies.
The researchers found that applicants who applied to multiple jobs screened by the same AI platform were more likely to be rejected by every employer than would be expected if each company made decisions independently. One in 10 applicants who submitted four applications through the platform was rejected by all four employers.
The takeaway
The study reinforces a key legal truth: employers remain responsible for the tools they use.
Even when using systems from third-party vendors, courts are likely to hold employers accountable for discriminatory outcomes. As a result, employers should treat AI governance as acentral part of their compliance efforts.
Employers that use AI in recruiting may want to consider the following safeguards:
- Conduct adverse-impact analyses by individual job position.
- Require vendors to provide validation studies and bias-testing data.
- Maintain human review of applicants screened out by AI.
- Periodically audit hiring outcomes for potential disparities.
- Document how AI systems are selected, tested and monitored.
- Establish oversight teams that include HR, legal and technology personnel.
- Train managers on the limitations and risks of automated decision-making.
- Closely monitor evolving federal and state regulations governing AI in employment.