The firm in question is Australia-based PredictiveHire, founded in October 2013. It offers a chatbot that asks candidates a series of open-ended questions. It then analyzes their responses to assess job-related personality traits like drive, initiative, and resilience. According to the firms CEO, Barbara Hyman, its clients are employers that must manage large numbers of applications, such as those in retail, sales, call centers, and health care. As the Cornell study found, it also actively uses promises of fairer hiring in its marketing language. On its home page, it boldly advertises: Meet Phai. Your co-pilot in hiring. Making interviews SUPER FAST. INCLUSIVE, AT LAST. FINALLY, WITHOUT BIAS.

As weve written before, the idea of bias-free algorithms is highly misleading. But PredictiveHires latest research is troubling for a different reason. It is focused on building a new machine-learning model that seeks to predict a candidates likelihood of job hopping, the practice of changing jobs more frequently than an employer desires. The work follows the companys recent peer-reviewed research that looked at how open-ended interview questions correlate with personality (in and of itself a highly contested practice). Because organizational psychologists have already shown a link between personality and job hopping, Hyman says, the company wanted to test whether they could use their existing data for the prediction. Employee retention is a huge focus for many companies that we work with given the costs of high employee churn, estimated at 16% of the cost of each employees salary, she adds.

The study used the free-text responses from 45,899 candidates who had used PredictiveHires chatbot. Applicants had originally been asked five to seven open-ended questions and self-rating questions about their past experience and situational judgment. These included questions meant to tease out traits that studies have previously shown to correlate strongly with job-hopping tendencies, such as being more open to experience, less practical, and less down to earth. The company researchers claim the model was able to predict job hopping with statistical significance. PredictiveHires website is already advertising this work as a flight risk assessment that is coming soon.

PredictiveHires new work is a prime example of what Nathan Newman argues is one of the biggest adverse impacts of big data on labor. Newman, an adjunct associate professor at the John Jay College of Criminal Justice, wrote in a 2017 law paper that beyond the concerns about employment discrimination, big-data analysis had also been used in myriad ways to drive down workers wages.

Machine-learning-based personality tests, for example, are increasingly being used in hiring to screen out potential employees who have a higher likelihood of agitating for increased wages or supporting unionization. Employers are increasingly monitoring employees emails, chats, and other data to assess which might leave and calculate the minimum pay increase needed to make them stay. And algorithmic management systems like Ubers are decentralizing workers away from offices and digital convening spaces that allow them to coordinate with one another and collectively demand better treatment and pay.

None of these examples should be surprising, Newman argued. They are simply a modern manifestation of what employers have historically done to suppress wages by targeting and breaking up union activities. The use of personality assessments in hiring, which dates back to the 1930s in the US, in fact began as a mechanism to weed out people most likely to become labor organizers. The tests became particularly popular in the 1960s and 70s once organizational psychologists had refined them to assess workers for their union sympathies.

In this context, PredictiveHires fight-risk assessment is just another example of this trend. Job hopping, or the threat of job hopping, points out Barocas, is one of the main ways that workers are able to increase their income. The company even built its assessment on personality screenings designed by organizational psychologists.

Barocas doesnt necessarily advocate tossing out the tools altogether. He believes the goal of making hiring work better for everyone is a noble one and could be achieved if regulators mandate greater transparency. Currently none of them have received rigorous, peer-reviewed evaluation, he says. But if firms were more forthcoming about their practices and submitted their tools for such validation, it could help hold them accountable. It could also help scholars engage more readily with firms to study the tools impacts on both labor and discrimination.

Despite all my own work for the past couple of years expressing concerns about this stuff, he says, I actually believe that a lot of these tools could significantly improve the current state of affairs.

Read more:

An AI hiring firm promising to be bias-free wants to predict job hopping - MIT Technology Review