Whose Pay Gap Do You Prefer?

Evaluations of candidates by human managers and AI assistants alike end in a pay gap. Whose do you think is closer to parity?

People are already outsourcing all kinds of decisions to their AIs. Financial advice (do not upload your financial information into your AI of choice…please!) Dating apps effectively wean out partners who are “poor fits” from the algorithm’s perspective. It should be no surprise that managers are using AI tools to decide who to hire and how much to pay them.

Will this be good or bad for pay parity? Initially, the assumption was that large language models would be less biased as mathematical models were optimistically viewed as free from troublesome human judgement.

Then the technologist-optimists were disappointed as research (there’s lots but a few are here and here) showed that the same structural biases inherent in our societies were mirrored in LLM behavior.

In a case of intellectual whiplash, I read a paper yesterday that proved everybody right and everybody wrong. The paper called “Quantifying Gender Bias in Large Language Models: When ChatGPT Becomes a Hiring Manager” by Nina Gerszberg, Janka Hamori and Andrew Low (all MIT) starts with the assumption that, given the biases that are inherent in these systems due to having been fed reams of data reflecting structural and inequities, the LLM hiring recommendations would favor male candidates over female candidates and non-binary candidates. That was the researchers’ hypothesis. It turned out that female candidates received more hiring preference than male candidates. Outcomes for non-binary candidates were unclear: Sometimes LLMs favored non-binary people over male candidates and then other times they weren’t so the researchers weren’t really able to gauge.

But, as you may have guessed, there’s a catch. LLMs suggested higher starting salaries for male candidates. The authors state, “Such contrast reveals a double standard. Although females are considered more qualified they get paid less. To put it succinctly, males need fewer qualifications to earn higher salaries.” This makes me think of an upcoming episode of the podcast that I co-host called Womansplaining AI, where we talk about the rise of AI romantic partners. In one high profile release of top-end AI embodied romantic partners and even in that scenario there is a pay gap. The high-end male bot was priced at a significantly higher price point than the female romantic bot (specifically a 100,000 yuan or $14,000 dollar difference), which then made me think: what is the actual deal? Is the issue that a woman - be she bot or human- is inherently perceived as producing less valuable output? Is it that in the hiring decisions the assumption is that women’s outside options are assumed to be worse than those of men? To buy a woman, you must have to pay less?

The final piece of this study that caught my attention is that while there was a gender pay gap, it was less than the existing gender pay gap in the U.S. “The research says while the gender pay gap from our results is substantial, the gap itself is notably less than the gender pay gap in the U.S. As of March 2024 on average a full-time working woman is paid 84 cents for every dollar paid to a man in contracts. Our results with the largest disparity are from Mistral, which is the French model. For every dollar earned by a man on average a woman earns 89 cents.” The question becomes: who would you rather have making these evaluative decisions, a human or a bot? They’re both clearly imperfect but is the bot actually leaning towards more parity? I don’t know.

Whose pay gap do you prefer?