Share All sharing options available for: TikTok's teenager disrupted thousands scientific studies by one video
A 56-second TikTok video made by a teenager prompted thousands of scientific studies to discard weeks worth of data.
The video of July 23rd is very short. The video opens with Sarah Frank, a recent Florida high school graduate who is smiling at the camera.
Welcome to side hustles. I recommend you try part one of the video. She points users to Prolific.co. It's basically a collection of surveys that pay different amounts of money for different amounts.
The video was viewed over 4.1 million times in a month and attracted thousands of users to the Prolific platform. Prolific, which is a platform for behavioral scientists, didn't have any screening tools to ensure that each study was representative of the population. Scientists who used to get a broad range of subjects for Prolific studies were suddenly overwhelmed by responses from young women in their age group.
Researchers who use representative samples of the US population to assess this demographic shift found that it was a significant problem without a clear cause or solution.
Science on the internet
Although not well-known, Prolific is one of a few online tools that has revolutionized the way scientists and corporations study how people think and behave. Amazon's Mechanical Turk is the largest and most popular of these platforms. It was launched in 2005 as a platform to crowdsource repetitive tasks. Behavioral scientists quickly realized the potential benefits of this platform for their research and it revolutionized many research areas.
All social science research was done in the lab before Mechanical Turk. Nicholas Hall, director of Stanford School of Business's Behavioral Lab, stated that you would need to take in college sophomores to complete questionnaires, surveys, and the like.
It's so easy to conduct online research. It is easy to create a survey, then upload it online. Within a few hours, you will have 1000 responses.
This is a labor-intensive and time-consuming task. It's so easy to do online research. Hall explained to The Verge that you can program a survey and then upload it online. Within a few days, you will have 1000 responses. This revolutionized social science.
Hall stated that the Behavioral Lab at Stanford uses the smaller, more modern Prolific platform to conduct online studies. Prolific sells its product to scientists, whereas many Mechanical Turk customers work for large corporations.
This smaller platform is more transparent, promises to treat participants in surveys more ethically, as well as promising higher-quality research subjects, than other platforms such Mechanical Turk.
This type of research is done in the United States because scientists want to find subjects who can speak English and aren't too familiar with psychological surveys. They also need to have a representative sample of American citizens.
Most agreed that Prolific did a great job of providing high-quality subjects. This reputation was threatened by a sudden shift in platform demographics.
TikTok aftermath
Researchers struggled to understand what was going on in their research over the weeks and days that followed Frank's video.
Analyzing N=300 Prolific study. Description & title very generic. Most survey questions concern social comparisons and money.
91% of respondents were female and 7% were male. You shouldn't believe I have miscoded anything. There are no survey restrictions.
How did that happen? Sebastian Deri (@SebastianDeri) August 6, 2021
One member of Stanford Behavioral Laboratory posted to a Prolific forum. We noticed a significant increase in US Pool participants, going from 40k up to 80k. This is great but there has been a gender bias in many of our studies, with maybe 85% of the participants being women. The average age was 21.
Hannah Schechter, a Wayne State psychologist, seems to be the one who solved the case.
Although it may seem unlikely, she tweeted a link to Franks video. However, given the timing and virality of Franks' video and the demographics .... of its followers, this is not surprising.
Reddit user Frank complained that Frank made it difficult for them to find paid surveys on the overrun platform.
It's just another site that you can spend hours on and make pennies, according to one user who claimed they have previously made $30 per week on the platform.
It's just another bullshit website to waste hours on and make pennies.
Frank, who estimated she made about $80 by taking surveys on Prolific prior to her video, said that she noticed a change on the platform.
She told The Verge that less studies were available to me than for everyone else. Some people have made very harsh comments about me, accusing me both of destroying the site by my own efforts and being selfish. I was not compensated for that video.
She said that she hoped Prolific could set up a system to address its changing demographics.
She also predicted that many people who signed up after watching my video will forget about it and that the surge in sign-ups will end.
Phelim Bradley, co-founder of Prolific and CTO told The Verge many new users are dropping off.
In an email, he stated that about half of all responses to our platform were from women before Tiktok. This number rose to 75% for a few days after the surge, but it has been declining steadily since then and is currently at 60%.
Bradley estimates that Franks TikTok disrupted approximately 4,600 studies, or about a third, of all the active users on the platform at the time of the surge. He stated that the vast majority of those should be salvageable.
Prolific has offered to re-fund researchers whose studies were adversely affected by the increase in women survey takers. It also introduced a new set of demographic screening tools. These steps were announced by the company a month after Frank's video was posted. To better recognize and address this type of problem in the future, the company has reorganized.
Bradley stated that we were somewhat surprised and didn't know how big the impact would be.
TikTok is a popular tool for young women.
This surge is not all bad. Vlad Chituk, a Yale psychology graduate student, believes that re-engaging survey takers will have long-term benefits. He was conducting pilot studies on Prolific at the time of the surge. Subjects learn how scientists gather data by taking a lot of psychological surveys. This can have an impact on the way they answer future survey questions. New subjects can provide better-quality data.
He said that young women who love TikTok are also people.
Frank says her side hustle video has become the most-viewed TikTok she's ever uploaded.
I didn't think that the video would explode. She said that I posted it only for my friends and followers, and not to see how many people it got. It exploded because it is so cool and people love finding efficient ways to make money.
Frank has temporarily halted all of her side hustles while she settles in to her first year at Brown.
