It was supposed to be a way to make extra money. When she was still in college, Oskarina signed up for Appen, an artificial intelligence data-labeling platform, to find a well-paid position in the oil industry.
Venezuela's economy tanked after that. A stable job was no longer an option because of inflation. Her side gig was temporary, but she would be full time in the future.
One of the millions of people who have left their country in search of better opportunities is a man named Fuentes. She is trapped at home because of a chronic illness that develops after delayed access to health care and because of opaque algorithms that dictate when she works and how much she earns.
She went on the record despite Appen threatening to retaliate against her. She wants people to understand what it's like to be a part of the global Artificial Intelligence development process, yet for the beneficiaries of her work to also mistreat her and make her invisible. She wants people to see the work that is being done.
Appen is one of many companies that offer data-labeling services. If you've bought groceries on Instacart or looked up an employer on Glassdoor, you've benefited from such labeling behind the scenes. Deep learning, an artificial intelligence technique that relies on scores of labeled examples to expand its capabilities, is one of the most profit-maximizing algorithms used in e- commerce sites, voice assistants, and self-driving cars.
There is a need for cheap labor to manually tag videos, sort photos, and make audio recordings. The market value of coordinating ghost work is projected to reach $13.7 billion by the year 2030.
There is a person named Juan Toro.
Venezuela has become a primary source of this labor over the last five years. The country plunged into the worst peacetime economic catastrophe in nearly 50 years as demand for data labeling exploded. Droves of well-educated people who were connected to the internet began joining crowdworking platforms as a means of survival.
The rise of the data-labeling industry was like a freak coincidence, according to a professor at the University of Applied Sciences.
The crisis in Venezuela has been a boon for these companies, which have gained some of the cheapest labor ever available. The rise of this fast-growing new industry in her country has been a mixed blessing. It has been a lifesaver for those without other options. Corporations have lowered their pay, suspended their accounts, or discontinued programs in an ongoing race to offer increasingly low-cost services to Silicon Valley, leaving them vulnerable to exploitation.
There are huge power imbalances, according to a PhD candidate at the University of Toronto who studies data annotators in Latin America. The rules of the game are made by them.
The arrangement is reminiscent of a colonial past when empires exploited the labor of more vulnerable countries and took advantage of the resources they needed to grow and develop.
The model could continue to spread as platforms look for even cheaper pools of labor in other countries. What began in Venezuela created a framework for how to meet the prices that clients have come to rely on, as well as setting an expectation for how little they should have to pay for such services.
The Venezuela example shows how a mixture of poverty and good infrastructure can make a situation like this possible.
The data-labeling industry exploded because of the old-school auto giants.
German car manufacturers were worried that the upstarts of the world would bring down their businesses. Legacy companies write blank checks to keep up when there is fresh-faced competition.
The self-driving car was the tech innovation of choice. The auto giants began pouring billions into their development, pushing the needs for data annotation to new levels.
Self-driving cars need millions of labeled examples to be taught, and they come in the form of hours of video footage. Self-driving cars need the highest levels of accuracy to work. Life and death can be decided by one too many mislabeled frames.
Amazon's crowdworking platform, MTurk, had reigned supreme for over a decade. It was launched in 2005 as a way for companies to get low- wage labor. MTurk was a general platform that produced varied results and couldn't guarantee a baseline of quality.
There is a person named Juan Toro.
A new generation of specialized crowdworking platforms emerged in the early 2010s, seeking to ensure better accuracy with a more hands-on approach to both the clients and workers. In order to get better performance, the automakers wanted accuracy of 99% or more. The specialized platforms took over after MTurk fell out of favor. Appen was adapted to the newer approach.
Scale AI is one of the most notable companies among the new specialized firms. It was founded in 2016 by a 19-year-old student at MIT, and it quickly amassed tens of thousands of workers and signed on big-name clients. If you could pull a rickshaw or label data in an air-conditioned internet cafe, it would be a better job, according to Mike Volpi, a general partner at Index. The scale is now worth $7.3 billion. In February, it was selected among several companies to provide services to the US Department of Defense.
Scale's early growth was dependent on its ability to provide high-quality labeled data quickly and cheaply. It launched a worker-facing platform to build a global pool of cheap contractors.
Scale first runs client data through its own systems to produce preliminary labels before posting the results to Remotasks, where human workers correct the errors. According to company training materials reviewed by MIT Technology Review, the company sends the data straight to the platform for others. One layer of human workers takes a first pass at labeling and then another reviews the work. Each worker's pay is tied to their speed and accuracy, which encourages them to complete tasks more quickly yet fastidiously.
Scale contractors initially sought in the Philippines. Both were natural fits, with histories of outsourcing and low wages. Around the same time, competitors such as Appen, Hive Micro, and Spare5 began to see a rise in signups from Venezuela. By the end of the year, an estimated 200,000 Venezuelas had registered for Hive Micro and Spare5 making up 75% of their respective workforces.
To support MIT Technology Review's journalism, please consider becoming a subscriber.
Scale followed its competitors into Venezuela. After seeing an increase in signups from the country, company executives saw an opportunity to turn one of the world's cheapest labor markets into a hub for its most intensive lidar annotations. It began aggressively recruiting workers from Venezuela using referral codes and a social media marketing campaign that made people believe they could make a lot of money.
The timing was perfect. The year before, the company restricted access to Spare5. The labelers moved to Remotasks in droves. In early 2020 Scale created a Venezuela-specific landing page for Remotasks and pushed users to join a new initiative called Remotasks Plus. The invitation-only program promised participants a new opportunity to receive more training, increase their earnings through minimum hourly wages and bonuses, and possibly advance inside the company.
The program's membership numbers began to go up after the global Pandemic began. Scale's foothold in the country was established by the new scheme. Scale was the go-to choice among prominent startups, Appen was the go-to choice among tech giants, and Hive Micro was the go-to choice among low-end clients.
The town where Fuentes lives is nestled in the mountains, an hour-long drive south of the country's budding tech hub. She shares an apartment with her husband, mom, aunt, uncle, and grandma, as well as her two dogs.
Her mom's hair salon is in the space. A woman with pink and lavender hair is given a haircut by her mom in the kitchen next to three other family members cooking lunch. The smaller of the two dogs, sporting a pink tutu and matching collar, settles down by the feet of the other dog. There are paper flowers on the walls.
There is a person named Juan Toro.
A browser shows her a running queue of tasks. Each displays a title and an anonymous client ID, as well as the number of units it is divided into and how much she can earn.
The tasks range from image tagging to content moderation to product categorization. The last task type has become so familiar that it's no longer necessary to translate the text from English to Spanish. For others, she uses a translation.
The system presents the client's instructions after she clicks in to claim a task. Sometimes they are clear and sometimes they are not. Sometimes there are no at all.
Her screen fills with a satellite image of a heavily forested area. There is no instructions, just a key that says "not trees" and a cursor that says "she should be outlining the corresponding parts of the image." Her answer has been rejected every time. She believes the client wants every tree to be outlined individually.
As she completes a few of the easier tasks, her earnings in the top right corner go up in pennies. She must convert the money into local currency if she wants to withdraw it. In Venezuela, most places don't accept payments from electronic wallet, and the black market to exchange them for local currency is filled with scam artists. At least she can use a payment service.
She opens up another task she hasn't been able to complete, this time in content moderation, and not for a client but as an assessment. Content moderation work is usually higher paying if she passes.
Do these social media videos contain any crime or human rights violations? There are a number of video players with radio buttons.
The name of the social media platform was redacted byMIT Technology Review after Appen's Christina Golden said the company could punish its client if it appeared in the story.
The video players are broken and show up blank. It is a bug, but her past experiences with Appen customer service suggest it is not worth the trouble to alert them. They are not clear and have a lot of slang. The task is impossible again.
At a time when the state-owned oil company in Venezuela was generating wealth, Fuentes studied oil and gas engineering in college. She was a good student and got an internship and a full-time job. She was on her way to Venezuela.
The economy was already collapsing by the final year of her master's program. The country's nearly complete dependence on oil revenues was threatening its decline.
She squirreled away $10 to $15 a week during breaks from writing her thesis because she knew that this one actually pays. She used an educational laptop that the government gave to kids a few years ago. The laptops that were used for such programs have since been sold and used by adults trying to access the digital economy.
The crisis had gotten worse by graduation. Her return offer no longer covered basic living expenses, but there were no better jobs for students leaving university. She wasn't sure if her family would be able to afford food if they stayed in the country.
She and her husband crossed the border in the early part of the year with only enough money for a week of groceries. In the same way as she was searching for stability, her family left for Venezuela to escape a different crisis.
The plan was to start fresh. They faced constant reminders of their precarious situation. They had a misunderstanding with their landlord. As her husband struggled to get work authorization, her new employer announced that it would be closing.
There is a person named Juan Toro.
She barely thought twice when she began to experience physical pain, believing it would pass once the turmoil was over. She was in the hospital for five days after starting a call center job.
The doctor warned that acute diabetes would kill her. She lost her vision for a month after she suffered disabling cramps. She immediately thought of how they would pay for her medication. She pulled out her old laptop and started working on Appen full time.
Wilson Pang, Appen's chief technology officer, said the company adjusts its pay per task to the minimum wage of each worker's locale. She could now rest more and take better care of herself, which included adhering to an intensive treatment regimen. She invested in a more powerful laptop to get higher paying tasks like 3D lidar labeling for self-driving cars. She made back the up-front costs.
As she remembers this part of the story, she smiles. With her husband employed and her earnings on Appen averaging $70 a week, she could finally breathe without worrying about money. She felt she had reached the end of a long and sunless tunnel when she was on those good days.
Leaving the country was not feasible for most of the other people in the country. The people who turned to data annotations did so because of a wave of crime that trapped them in their homes.
The platforms became the full-time focus of many families. Sometimes parents and children took turns on a shared computer, other times women took care of household chores so that the men of the household could work.
The window of opportunity was getting smaller. As more and more workers joined the platform, the number of tasks on Appen began to dwindle. She says that the task queue used to be populated 24 hours a day. Work arrived at odd hours and was increasingly empty.
Users who joined later weren't so lucky. Appen divided its accounts into four levels. Users needed to complete tasks on levels 0 and 1 to be able to access additional jobs on levels 2 and 3.
The creators of new accounts received negligible amounts of money because lower-level tasks became nearly extinct. The only way to break in was to buy an existing high-level account in an underground market, but those who did so risked having their accounts shut down for violating company policy.
Golden says it has moved away from a level-based model, but its projects still have specific qualifications and are not open to everyone.
Remotasks was the next best option. Hive Micro is the easiest service to join, but it offers the most disturbing work, such as labeling terrorist imagery. The system started to show cracks after Remotasks Plus was launched. Many users realized that their hours were being undercounted, which lowered their earnings. They were held to higher standards, with greater risk of suspension for not being fast or precise enough.
“I realized that their approach was to drain each user as much as possible."
Ricardo Huggines, a Venezuelan worker who joined Remo Plus to support his wife and kids
Matt Park, the senior vice president of operations at Scale, says Remotasks invests heavily in training and support for all taskers. Workers found that there wasn't enough support to help people meet standards.
Remo Plus capped earnings for people who worked over 60 hours a week. Scale continued its publicity campaign, posting videos to YouTube, Facebook, and Instagram with testimonials and attractive stock footage showing stacks of US dollars.
A university student who worked on Remo Plus and asked to remain anonymous for fear of retaliation said that they were lying when they said stability was a long-term job.
When workers experienced frustration, they were unable to address it. Trainers, community managers, and recruiters were the main channels of communication for the company.
The people had no incentive to advocate on their behalf. Workers who asked questions were mistreated. The former computer engineer was kicked out of Remo Plus after he was too vocal about reduced payments and increased workload.
Park says that they take all worker complaints seriously and investigate allegations.
The program became more disorganized as time went on. People were left with incomplete tasks for which they were later punished because of the platform's bugs. At one point, Scale switched from PayPal to the digital wallet Air, which supported bolivares, the local currency, in order to move money into Venezuela. Workers in Scale's original set up for Venezuela complained about their payments being delayed for weeks or even months.
Scale squeezed workers' earnings even more at the start of the 21st century. Everyone was migrated back to the standard Remotasks platform after it shut down Remo Plus. Many workers say they never received their final payouts, but company records show no outstanding payments or pay related support inquiries.
Some workers heard that the company closed the program for people who took advantage of it. Scale told workers that the program was over and that it had been an experiment.
The whiplash disrupted their livelihood and their family's means of survival.
Fuentes waits impatiently by her computer, ready to start tasking at a moment's notice. Some weeks, her hypervigilance leads to nothing, others she brings in a dismal $6 to $8, falling short of the threshold to withdraw her money. She makes $300 in a few hours when she gets a high-paying job.
Her average income is tenable because of the windfalls. They are rare enough to keep her connected to her computer. She can't afford to lose the opportunity if a good task appears. She missed a task that would have made her $100. She learns that clients usually post tasks during their working hours, so she restricts her walks to weekends.
She uses Telegram and Discord to vent her frustration. Members trade strategies and hacks to increase their earnings. Tools developed by the community are shared by them. The browser extension that sounds an alarm when a new task appears is one of the tools that Fuentes uses. She keeps it on so she can wake up in the middle of the night.
There is a person named Juan Toro.
One group has helped her a lot. Appen sends different work to different workers based on their location, speed, and ability. The people in the group don't know the exact mechanism, but they know they get different tasks. As work on Appen began to dwindle, they realized they could access each other.
The tasks are pooled together. Whenever a task appears in one member's queue, that person copies the task-specific URL to everyone else. Even if the task never showed up in the queue, anyone who clicks it can claim it as their own. The system is not perfect. When multiple members claim the same task in parallel, each task has a limited number of units, which disappear faster. As long as she clicks the link before it goes away, the platform will let her complete whatever units are left, and Appen will pay.
The group keeps a record of which client IDs should not be used. A devastating suspension account can be caused by some clients who are particularly harsh in grade performance. Almost every member of the group has experienced at least one. When it happens, you lose your access to any earnings that haven't been withdrawn.
When she appealed, customer service confirmed that it was an administrative error. It took months for her account to be restored, but she was able to use the internet to communicate in English.
Golden says Appen has seen an increase in workers engaging in acts it considers fraud, such as using a virtual private network to locate themselves in higher-earning countries, which is why the company looks for these behaviors and shuts down accounts deemed illegitimate. Workers say it's the platform's unrealistic expectations and policies that push them to find creative solutions.
Conditions on Remotasks have gotten worse since the shutdown of Remo Plus. Workers say the platform is buggy and sometimes misleading, while payments have become more unreliable. Some people can spend hours completing tasks and only get a small portion of the total amount. The platform is designed to save tasker work throughout the process.
“They treat us like we’re not human.”
Hossam Ashraf Esmael, a former North Africa-based community manager at Remotasks
As Remotasks has continued its global expansion, annotators in Venezuela have become suspicious that they are being treated differently from their counterparts in higher-income countries. In North Africa, where the platform has expanded in the last two years, annotators say that Scale has reduced their pay by more than a third in a matter of months and even taken away earnings, leaving some workers with negative pending payments.
The Filipino and European annotators that the workers in Venezuela and North Africa speak to have never experienced the same treatment.
Scale tried to keep workers from resisting the changes. When a group of North African annotators tried to fight pay cuts, they were retaliated against. The company threatened to ban anyone engaging in protests and revolutions, and eight workers were at risk of having their accounts shut down to speak about their experiences. The workers say Scale created a new quota system that removes workers from the project if they don't complete a certain number of tasks within a given time. Around 20 of them have already been kicked out.
They treat us like we're not human, says Hossam Esmael, a former community manager at Remotasks.
In February, pay rates for this project were updated to match the average payments for similar Remotasks projects.
There is a person named Juan Toro.
The MIT Technology Review created a account in Venezuela to corroborate workers' testimonies. The experience was very difficult. Task instructions were hard to understand. There is a timer at the top left of the screen that does not have a clear deadline or a way to stop it. If a worker leaves a task incomplete for too long, a timer will return it to the pool for someone else to claim. We went back to the instructions page because of three mistakes. Sometimes the platform wouldn't load.
The training materials showed a picture of a woman showering. If you make high-quality annotations and follow the rules of the project, you can get a high compensation. Workers in Venezuela earn an average of 90 cents an hour, according to Park.
Simala Leonard, a computer science student at the University of Nairobi who studies artificial intelligence and works on Remotasks, says the pay for data annotators is unfair.
The most fundamental part of machine learning is paid a pittance.
Data-labeling companies have sought to establish a higher standard for working conditions in parallel with the rise of platforms like Scale. They offer stable wages and benefits, good on-the-job training, and opportunities for career growth and promotion.
A PhD candidate at the Technical University of Berlin says that the model may improve the lives of 50 workers.
The players are willing to race to the bottom. To keep their prices competitive, the firms similarly source workers from impoverished and marginalized populations who remain vulnerable to exploitation.
During the Pandemic, some companies loosened their standards. The market's sudden oversupply of labor drove down the average cost of data annotation as they lowered their wages and worked longer hours. It has affected employees like Jana, who asked us not to use her real name because her diminishing income no longer supports her child. She has two jobs. She works full time at a firm that is seen as a pioneer in ethical data labeling. She works from 3 a.m. until morning on Remotasks. She says you just hope for better days.
Those better days won't come without coordinated international advocacy and regulation to limit how low the industry can go. If not the Philippines, then Venezuela. If not Venezuela, then somewhere else.
Scale has expanded well beyond Venezuela. Virtual boot camps were offered in Asia, Latin America, sub-Saharan Africa, and the Arabic-speaking countries. According to traffic data from traffic analyzer Semrush, the proportion of logins from Venezuela is falling.
Data from web advertising shows that it has been conducting in-person boot camps in the country.
Appen could also abandon her. She is grateful despite the stress and hardship it has caused.
One of the reporters on the article poses with Fuentes.
She hopes Appen can become a more traditional employer in four to five years.
Golden says that they are proud of their contributors and are working hard to improve internal processes to make it a better experience for them.
As the sun begins to set, Fuentes asks her uncle to take a photo. As she cuddles her dog, her smile beams through her mask. She wants people to know her name and face after serving the platform and its clients as an anonymous worker for many years.
A few weeks later, she sends the photo with a message: “Don’t forget us,” it says.
All Rights Reserved © 2024
