AI could be the key to ending discrimination in hiring, but experts warn it can be just as biased as humans

• Employers are increasingly turning to artificial intelligence-driven tools to carry out recruitment and hiring. Companies like Amazon, FedEx, Target, and Capital One have tested or used AI hiring software.
• These tools sift through heaps of resumes quickly, evaluate candidates' answers to written questions and interactive games, and conduct video interviews with candidates.
• Vendors tout AI hiring tools as a way to root out discrimination, but experts and activists warn that AI-driven hiring tools are just as biased as the humans who train them.
• Visit Business Insider's homepage for more stories.

The next time you apply for a job, the fate of your application may not be in human hands - instead, an algorithm could determine whether or not you make the cut.

Employers are increasingly turning to artificial intelligence-driven tools to carry out recruitment and hiring. Target, Hilton, Pepsi, Amazon, and Ikea are among the companies who have tested or used algorithms to determine who to hire, and the list is growing across the employment spectrum from low-wage jobs to white-collar positions.

Advocates say AI is the key to rooting out human bias and ending discrimination in the hiring process. But its critics warn that AI-driven hiring tools are just as biased as the humans who train them.

Predictive hiring tools are mostly used to weed out candidates deemed unfit for hire, rather than affirmatively choose who gets a job. These tools sift through huge numbers of resumes in short time periods, evaluate candidates' answers to written questions and interactive games, and conduct video interviews with candidates.

Read more: How AI is changing everything

Companies who sell the AI tools say they expect that the job of human recruiters will soon be entirely replaced by robots in some sectors, and investor interest in these vendors is growing. Researchers at Cornell found that there are at least 19 companies providing AI-driven hiring tools worldwide, with funding ranging from $1 million to $93 million per vendor.

But AI experts have raised concerns about the technology's rapid growth.

"Technology is rapidly changing how people find jobs, learn about jobs, apply to those jobs, and are evaluated for those jobs," Aaron Rieke, managing director at tech equity nonprofit Upturn, told Business Insider. "The fear is if that is not done very carefully and thoughtfully, there could be problems down the road."

Meanwhile, AI-driven hiring tools haven't drawn much scrutiny from regulators and lawmakers. However, the House Education and Labor Committee is holding a series of hearings on the impact of AI on the workforce this fall, which could be a precursor to legislation.

AI hiring tools could automatically discriminate against certain job applicants

Just like all algorithms, AI hiring tools are trained by humans, typically using data sets from the real world meant to help AI recognize the features of a "good" or "bad" candidate.

Accordingly, tech experts and activists have warned that AI hiring tools might pick up on preexisting human biases, especially in fields like tech, where progress towards diversity has been slow.

In a report published in December, Upturn researchers highlighted how these assumptions could lead to unintentional discrimination by AI hiring tools.

"What can naturally happen is you build a model that identifies common characteristics of your current workforce, which isn't diverse," Rieke said. "Or might reflect the fact that hiring managers have traditionally given preference to male candidates over women."

Employers also tend to blindly trust AI as an objective standard-bearer without questioning its potential for bias, according to Joy Buolamwini, a computer scientist based at the MIT Media Lab. Buolamwini founded the Algorithmic Justice League, which conducts research and advocacy on AI bias.

"Even if a company has good intentions, products that are trained on data collected from current employees deemed successful could learn discriminating characteristics that are due to identity traits instead of job competency," Buolamwini wrote in an email to Business Insider. "Data is not necessarily neutral."

Can AI be trained to root out human bias?

Vendors of AI-driven hiring tools argue the opposite, claiming that AI is the most surefire way to ultimately eliminate human bias in the hiring process.

Eric Sydell, the executive vice president of innovation at Modern Hire, told Business Insider he believes algorithms can be programmed with the specific goal of ruling out the unfair preferences that humans act on. Modern Hire sells a software that allows companies to put candidates through interviews and "virtual job tryouts," and is used by companies including Capital One, Allstate, SunTrust, and FedEx.

"AI has tremendous potential to make things more fair and more valid," Sydell said. "There's a million of those [biases] that humans are holding in their heads at all times … with AI we can eradicate human biases."

Rieke said it's too soon to tell whether AI-driven hiring tools are already working against bias, but he believes it's possible that such tools could one day prove less biased than human recruiters.

"The old-fashioned, human way of doing this is a process that has consistently yielded staggering bias against people," Rieke said. "I think there's the possibility for hiring tools to fairly and equitably hire people … but the devil's in the details."

NOW WATCH: We compared 13 veggie burgers and Burger King's Impossible Burger was the only one that stood out