Why a $14 Billion Startup Is Now Hiring PhD’s to Train AI From Their Living Rooms

Scale used to primarily hire cheap labor in Africa, India and the Philippines to label autonomous vehicle sensor data—such as images of pedestrians, stoplights and traffic signs—for firms such Alphabet’s Waymo and GM’s Cruise, or to tag shopping images for Meta’s Instagram. But lately, as money has started pouring into large-language models, Scale has shifted its focus to recruiting highly skilled contractors in the U.S. with the expertise to help train those models. About 300,000 of them take assignments through a Slack group run by Outlier, a Scale subsidiary.

Scale’s shift highlights a little-understood reality—that human contractors play a crucial role in developing the latest AI systems. “It’s not just a bunch of programmers in a project somewhere telling the AI what to do and how to do it,” Mattingly said. “There’s a great deal of laborious human work going on to help train models.”

The first step for training a LLM, called pretraining, involves analyzing terabytes of text using thousands of advanced chips, a process that requires millions of dollars in investment and weeks of waiting. But that’s not enough to ensure that systems like Anthropic’s Claude, OpenAI’s ChatGPT, Meta’s Llama and Google’s Bard deliver accurate answers written in humanlike style.

Accomplishing that requires a second step: fine-tuning, which involves armies of human contractors, either hired in-house or from Scale, other startups like Surge AI and Labelbox, or publicly traded Telus International. Those contractors essentially teach the AI models by writing their own ideal responses to questions submitted to chatbots.

Scale, founded in 2016, pioneered some of this work, partnering with OpenAI in 2019 to study a process called reinforcement learning through human feedback, which has become part of making chatbots generate humanlike responses. Scale now calls what it runs a “hybrid human-AI system to produce high-quality data at a low cost,” according to a presentation shared with investors in February, which was obtained by The Information.

Dan Levine, a partner at venture capital firm Accel and a Scale board member, said he likes to explain the problem using a simplified example: It is difficult for AI models to understand the meaning of idiosyncratic English phrases like “twist my arm” without seeing many examples. “What is artificial intelligence but literally allowing computers to artificially be like human intelligence?” he said.

“Right now, there’s an incredible amount of demand for what Scale provides, including among the most sophisticated companies on the planet,” he added.

Revenue to Triple

The shift toward LLMs has helped Scale jump-start growth in its business after it had started to slow down. Before the launch of ChatGPT sparked a flurry of investment in LLMs, Scale’s revenue growth had dipped below 50% in 2022, significantly behind previous revenue forecasts it had shared with investors in 2019, investor presentations show. The startup was losing money, spending more than $1.40 in operating costs for every $1 of revenue two years ago.

Thanks to its push into training LLMs, the company, run by 27-year-old CEO Alexandr Wang, has told investors it expects revenue to more than triple to just over $1 billion this year from $334 million last year, with minimal cash burn. Scale landed a $13.8 billion valuation last month from existing investors including Accel, as well as new corporate backers Amazon, Meta Platforms and Intel.

That growth comes at a cost. Scale has had to shift to a more highly educated workforce based in the U.S.—including PhDs, doctors and lawyers—sharply raising its labor expenses. Its gross margin, which includes its costs to pay contractors, fell from 59% to 49% between 2022 and 2023, financial data obtained by The Information show.

While its workers in Africa, India and the Philippines earned $1.50 an hour on average, according to a 2019 investor presentation, Scale has been paying $40 an hour to people like Melissa Quashie, a Massachusetts-based freelance writer and editor recruited through LinkedIn a few months ago to work for Scale as a contract employee.

The job at Scale felt like the “nerdiest videogame I ever played,” said Quashie. Her tasks include rating different responses generated by LLMs, based on how well the models were answering people’s questions and how well the answers were written.

She once spent two hours writing a three-day meal plan designed to help improve automated responses for another chatbot.

But her experience has been uneven. To feed the demands of customers like Meta and Alphabet, which want work done quickly, Scale has amassed a huge workforce. That has meant the assignments that Scale gets aren’t consistent enough to keep the contractors employed on an ongoing basis.

As a result, the work has often been unsteady and high pressure, even if the pay and flexibility are alluring, according to interviews with 10 contractors who work for Scale. Many complained about “empty queues,” or a lack of new projects to do. Quashie, for instance, said projects that had once flowed in regularly started to dry up.

“If you start to count on the money a little bit, you’re, like, ‘Where’s the work? Why aren’t you answering my Slack messages?’” she said.

Workers also complain about delayed payouts, failure to deliver promised bonus payments, poor training and a system that crashes frequently, causing them to lose their written work. Mattingly said he got kicked off the platform, without explanation from Scale. Several said the pay and flexibility the job provides is for the most part good enough to make it worth putting up with the problems. A Scale spokesperson, Amy Swanson, said in an email: “The program is flexible work, not a full-time job.”

Scale, meanwhile, has told investors it is trying to get costs down. The company has forecasted it would boost that gross profit margin 5 percentage points this year and then to 60% in 2025.

The company told investors it is making its human contractor force less costly by using internal tools to automatically identify “efficient experts” to train models, as well as relying on computer-generated data to augment work done by humans.

The company also emailed contractors in late April saying it was cutting pay for the work they performed training for tasks, which they need to do each time they start a new project. One U.S.-based worker said his new training pay was $17 an hour, according to a message viewed by The Information. Others said the company had cut pay for other projects, from $40 to about $22.

Data Foundry for AI

In an investor presentation, Scale said it is building crucial AI infrastructure. The company has started branding itself as a “data foundry” for AI, evoking a physical semiconductor plant. Wang has talked publicly about the benefits that Ph.D. holders, doctors and lawyers can bring to working on training AI systems.

“We need the best and brightest minds to be contributing data,” he said on a podcast last month. “Their work is how they have a very scaled society impact. There’s an argument you can make that producing data for AI systems has near-infinite impact.”

But several former Scale employees said the system to pay out workers was often buggy and outdated, and the company often prioritized features for tech company customers over improving its systems for workers. Contractors sometimes weren’t paid for their work unless customers verified it was done correctly. Workers who performed well at the tasks could make a decent living in low-wage countries, they said.

Virginia Puccio, who was a Scale customer as the head of data operations at AI-powered retail startup Standard Cognition, said her firm and other AI companies ended up relying on outsourced work from firms like Scale because the technology wasn’t advancing quickly enough. She and other technologists didn’t think about the human contractors on the other side of the tech.

“We didn’t know who they were using, how much they were paying, and I don’t think we thought to ask,” said Puccio, who now runs her own startup, Fuel AI.

As it has expanded in the U.S., Scale has pulled back from its overseas operations. The company recently shut down contractor outposts in Kenya, Nigeria and Pakistan, according to the publication Rest of World.

Swanson, the Scale spokesperson, said the company “consistently improves its pay systems to ensure its contributors are paid accurately and on time” and has an anonymous system in place to address their concerns. “Scale views an investment in Contributor experience as an investment in the quality of work we provide to our customers,” she wrote.

Jeff Wilke, a former top Amazon executive who invested in Scale in 2021 and advises Wang, said in an interview he didn’t “know as much about the exact kinds of projects that [Scale was] working on in the past. But the work now is about extremely complex, sophisticated answers to very technical prompts.”

That more technical work is “probably going to work out pretty well for the workers,” he added. “Prices typically follow scarcity. And if there aren’t that many people that can do it, the pay is gonna be great.”

Buggy and Unsteady

The company has tried to create some camaraderie among its temporary workforce by flying contractors to writing workshops of sorts for AI companies. It flew dozens of top Scale contractors to Austin, Texas, and Jacksonville, Fla., for dayslong sessions of rating responses and writing essays based on questions made to the AI systems.

One contractor who went to the Austin retreat said about 50 Scale contractors worked on a project they learned was for Alphabet’s Bard chatbot. They reviewed the responses each person wrote for different prompts, and sang karaoke together at night.

In Jacksonville, Quashie met college professors, doctoral students, screenwriters and podcasters. “We’d bang away for six hours, then have a glass of wine,” Quashie said.

“Everyone’s very excited about making LLMs better. What nobody talks about is, who’s going to lose their job because we’re doing this job?”