A Reading List On The Human Labor Behind AI, Machine Learning, Data Labeling, And Content Moderation

a reading list on the human labor behind AI, machine learning, data labeling, and content moderation

bringing a global labor perspective to the “ai is gonna steal our jobs!” discourse that usamerican creative workers don’t really like…

(based on this twitter thread)

Google’s AI Chatbot Is Trained by Humans Who Say They’re Overworked, Underpaid and Frustrated (12 July 2023)

“If you want to ask, what is the secret sauce of Bard and ChatGPT? It’s all of the internet. And it’s all of this labeled data that these labelers create,” said Laura Edelson, a computer scientist at New York University. “It’s worth remembering that these systems are not the work of magicians — they are the work of thousands of people and their low-paid labor.”

The Hidden Workforce That Helped Filter Violence and Abuse Out of ChatGPT (11 July 2023)

ChatGPT is one of the most successful tech products ever launched. And crucial to that success is a group of largely unknown data workers in Kenya. By reviewing disturbing, grotesque content, often for wages of just two to three dollars an hour, they helped make the viral chatbot safe. WSJ’s Karen Hao traveled to Kenya to meet those workers and hear about what the job cost them.

The workers at the frontlines of the AI revolution: The global labor force of outsourced and contract workers are early adopters of generative AI — and the most at risk (11 July 2023)

Since the blockbuster launch of ChatGPT at the end of 2022, future-of-work pontificators, AI ethicists, and Silicon Valley developers have been fiercely debating how generative AI will impact the way we work. Some six months later, one global labor force is at the frontline of the generative AI revolution: offshore outsourced workers.

Inside the AI Factory: the humans that make tech seem human (20 June 2023)

You might miss this if you believe AI is a brilliant, thinking machine. But if you pull back the curtain even a little, it looks more familiar, the latest iteration of a particularly Silicon Valley division of labor, in which the futuristic gleam of new technologies hides a sprawling manufacturing apparatus and the people who make it run.

OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic (18 January 2023)

OpenAI took a leaf out of the playbook of social media companies like Facebook, who had already shown it was possible to build AIs that could detect toxic language like hate speech to help remove it from their platforms. The premise was simple: feed an AI with labeled examples of violence, hate speech, and sexual abuse, and that tool could learn to detect those forms of toxicity in the wild. That detector would be built into ChatGPT to check whether it was echoing the toxicity of its training data, and filter it out before it ever reached the user. It could also help scrub toxic text from the training datasets of future AI models.

To get those labels, OpenAI sent tens of thousands of snippets of text to an outsourcing firm in Kenya, beginning in November 2021. Much of that text appeared to have been pulled from the darkest recesses of the internet. Some of it described situations in graphic detail like child sexual abuse, bestiality, murder, suicide, torture, self harm, and incest. … The data labelers employed by Sama on behalf of OpenAI were paid a take-home wage of between around $1.32 and $2 per hour…

The ‘Invisible’, Often Unhappy Workforce That’s Deciding the Future of AI (9 December 2023)

Among a range of conclusions, the Google study finds that the crowdworkers’ own biases are likely to become embedded into the AI systems whose ground truths will be based on their responses; that widespread unfair work practices (including in the US) on crowdworking platforms are likely to degrade the quality of responses; and that the ‘consensus’ system (effectively a ‘mini-election’ for some piece of ground truth that will influence downstream AI systems) which currently resolves disputes can actually throw away the best and/or most informed responses.

The Exploited Labor Behind Artificial Intelligence: Supporting transnational worker organizing should be at the center of the fight for “ethical AI.” (13 October 2022)

So-called AI systems are fueled by millions of underpaid workers around the world, performing repetitive tasks under precarious labor conditions. And unlike the “AI researchers” paid six-figure salaries in Silicon Valley corporations, these exploited workers are often recruited out of impoverished populations and paid as little as $1.46/hour after tax. Yet despite this, labor exploitation is not central to the discourse surrounding the ethical development and deployment of AI systems.

A factory line of terrors: TikTok’s African content moderators complain they were treated like robots, reviewing videos of suicide and animal cruelty for less than $3 an hour (1 August 2022)

“The devil of this job is that you get sick slowly — without even noticing it,” said Wisam, a former content moderator who now trains others for Majorel. … While TikTok does use artificial intelligence to help review content, the technology is notoriously poor in non-English languages. For this reason, humans are still used to review most of the heinous videos on the platform.

Human Touch: Artificial intelligence may be making some jobs obsolete but it has given a new lease of life to one group of people who play an unglamorous but critical role in the machine learning pipeline: first generation women workers in Indian towns and villages (20 July 2022)

“Any major technology company in the last 10 years has been powered by a throng of people … At some level, there’s denial. Investors like to hear that technology sells itself once you write the code. But that’s not really true.” … “Data work has a racial and class dynamic. It is outsourced to developing countries while model work is done by engineers largely in developed nations … Without their labour, there would be no AI.”

Desperate Venezuelans are making money by training AI for self-driving cars (29 August 2022)

How the AI industry profits from catastrophe: As the demand for data labeling exploded, an economic catastrophe turned Venezuela into ground zero for a new model of labor exploitation (20 April 2022)

Most profit-maximizing algorithms, which underpin e-commerce sites, voice assistants, and self-driving cars, are based on deep learning, an AI technique that relies on scores of labeled examples to expand its capabilities. … The insatiable demand has created a need for a broad base of cheap labor to manually tag videos, sort photos, and transcribe audio. The market value of sourcing and coordinating that “ghost work” … is projected to reach $13.7 billion by 2030.

Over the last five years, crisis-ridden Venezuela has become a primary source of this labor. The country plunged into the worst peacetime economic catastrophe facing a country in nearly 50 years right as demand for data labeling was exploding. Droves of well-educated people who were connected to the internet began joining crowdworking platforms as a means of survival.

Facebook Faces New Lawsuit Alleging Human Trafficking and Union-Busting in Kenya (11 May 2022)

“We can’t have safe social media if the workers who protect us toil in a digital sweatshop… We’re hoping this case will send ripples across the continent—and the world. The Sama Nairobi office is Facebook’s moderation hub for much of East and South Africa. Reforming Facebook’s factory floor here won’t just affect these workers, but should improve the experience of Facebook users in Kenya, South Africa, Ethiopia, and other African countries.”

Inside Facebook’s African Sweatshop (14 February 2022)

Here in Nairobi, Sama employees who speak at least 11 African languages between them toil day and night, working as outsourced Facebook content moderators: the emergency first responders of social media. They perform the brutal task of viewing and removing illegal or banned content from Facebook before it is seen by the average user. …

The testimonies of Sama employees reveal a workplace culture characterized by mental trauma, intimidation, and alleged suppression of the right to unionize. The revelations raise serious questions about whether Facebook… is exploiting the very people upon whom it is depending to ensure its platform is safe

Refugees help power machine learning advances at Microsoft, Facebook, and Amazon: Big tech relies on the victims of economic collapse (22 September 2021)

Microwork comes with no rights, security, or routine and pays a pittance — just enough to keep a person alive yet socially paralyzed. Stuck in camps, slums, or under colonial occupation, workers are compelled to work simply to subsist under conditions of bare life. This unequivocally racialized aspect to the programs follows the logic of the prison-industrial complex, whereby surplus — primarily black — populations [in the United States] are incarcerated and legally compelled as part of their sentence to labor for little to no payment. Similarly exploiting those confined to the economic shadows, microwork programs represent the creep of something like a refugee-industrial complex.

(an excerpt from the book Work Without the Worker: Labour in the Age of Platform Capitalism by Philip Jones)

AI needs to face up to its invisible-worker problem (11 December 2020)

A.I. Is Learning From Humans. Many Humans. (16 August 2019)

A.I. researchers hope they can build systems that can learn from smaller amounts of data. But for the foreseeable future, human labor is essential. “This is an expanding world, hidden beneath the technology,” said Mary Gray, an anthropologist at Microsoft and the co-author of the book “Ghost Work,” which explores the data labeling market. “It is hard to take humans out of the loop.”

[book] Behind the Screen: Content Moderation in the Shadows of Social Media by Sarah T. Roberts (June 2019)

Social media on the internet can be a nightmarish place. A primary shield against hateful language, violent videos, and online cruelty uploaded by users is not an algorithm. It is people. Mostly invisible by design, more than 100,000 commercial content moderators evaluate posts on mainstream social media platforms: enforcing internal policies, training artificial intelligence systems, and actively screening and removing offensive material—sometimes thousands of items per day

[book] Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass by Mary L. Gray and Siddharth Suri (May 2019)

Hidden beneath the surface of the web, lost in our wrong-headed debates about AI, a new menace is looming. … services delivered by companies like Amazon, Google, Microsoft, and Uber can only function smoothly thanks to the judgment and experience of a vast, invisible human labor force. These people doing “ghost work” make the internet seem smart. They perform high-tech piecework: flagging X-rated content, proofreading, designing engine parts, and much more. An estimated 8 percent of Americans have worked at least once in this “ghost economy,” and that number is growing. They usually earn less than legal minimums for traditional work, they have no health benefits, and they can be fired at any time for any reason, or none.

[follow-up articles about the book here and here]

Inmates in Finland are training AI as part of prison labor (28 March 2019)

“Prison labor” is usually associated with physical work, but inmates at two prisons in Finland are doing a new type of labor: classifying data to train artificial intelligence algorithms for a startup. … “The hook is that we have this kind of hype circulating around AI so that we can masquerade really old forms of labor exploitation as ‘reforming prisons,’… They’re connecting social movements, reducing it to hype, and using that to sell AI.”

How Crowdworkers Became the Ghosts in the Digital Machine: Since 2005, Amazon has helped create one of the most exploited workforces no one has ever seen (5 February 2014)

Crowdworking is often hailed by its boosters as ushering in a new age of work. With the zeal of high-tech preachers, they cast it as a space in which individualism, choice and self-determination flourish. … But if you happen to be a low-end worker doing the Internet’s grunt work, a different vision arises. According to critics, Amazon’s Mechanical Turk may have created the most unregulated labor marketplace that has ever existed. Inside the machine, there is an overabundance of labor, extreme competition among workers, monotonous and repetitive work, exceedingly low pay and a great deal of scamming. In this virtual world, the disparities of power in employment relationships are magnified many times over, and the New Deal may as well have never happened.