What researchers can actually see when you take a Prolific study
I got curious about how much researchers can actually tell about how we take their surveys, not just the answers we give them. I looked into Prolific’s rules and Qualtrics in particular, since so many studies use it, and there were a few things I didn’t know.
So I figured I’d share what I found.
1. You don’t have to match the estimated completion time.
If a study says 10 minutes and you honestly finish in 6, just finish it. You don’t have to sit there doing nothing for another 4 minutes so you don’t look suspicious.
Prolific only allows speed alone as a reason for rejection when you’re an extreme statistical outlier, 3 standard deviations below the average completion time.
2. Researchers may be able to see how long you spent on each page.
Qualtrics has a Timing feature researchers can add to individual pages. It can record things like how long the page was open before your first click, how long until your last click, how long before you hit Next, and even the number of clicks.
So ironically, racing through a 10-minute survey in 4 minutes and then sitting on the last page for 6 minutes to run out the clock could look stranger than simply submitting it when you’re done.
3. Straight-lining a Likert scale is not automatically a problem.
This was something I specifically wondered about. Say there are 20 questions and your honest answer really is “Agree” to all 20. Apparently the answer is simple: answer Agree to all 20.
Researchers can obviously see the pattern, but you shouldn’t start changing truthful answers just to make yourself look less suspicious.
The problem is straight-lining because you’re not reading. That’s when you run into reverse-worded questions or an attention check in the middle.
4. The same thing goes for other obvious patterns.
Going 1-2-1-2-1-2 down a page just to get through it isn’t any better. Researchers can see that kind of pattern too.
Basically, don’t try to manufacture what you think “normal” answers are supposed to look like.
5. Be especially careful with long repetitive blocks.
Those 20 or 30 nearly identical personality questions are probably where it’s easiest to go on autopilot.
I’m not saying slow down just to add time to the survey, I’m saying make sure you’re still actually reading the question instead of seeing the first few words and assuming you know what it says.
That’s also where an attention check can get you.
6. Attention checks have rules.
Researchers can’t just throw in any random gotcha and use it to reject somebody.
For studies under 5 minutes, failing one valid attention check can be enough. For studies 5 minutes or longer, Prolific generally requires you to fail multiple valid attention checks.
7. Comprehension checks are a little different.
If they’re testing whether you understood instructions you need in order to do the study, you’re generally supposed to be allowed to reread those instructions and get another attempt.
If you still don’t understand them, you may be asked to return the study. That’s different from simply rejecting your submission.
8. Researchers can decide your data isn’t useful without having the right to reject you.
I think this is an important distinction.
Maybe their experiment was designed to make you feel a certain way and it didn’t. Maybe your answers don’t fit what they’re trying to measure. That may give them a reason not to include your data in their analysis.
It does not automatically give them a reason to reject you and not pay you.
“Your response isn’t useful to my experiment” and “you didn’t make a good-faith effort” are two different things.
9. Tab switching can sometimes be detected.
Prolific now has authenticity checks that can work with Qualtrics, particularly with free-text questions, and tab switching is one of the behaviors they can use as a signal.
That doesn’t mean the researcher can see what was in the other tab. They can potentially tell that you left the survey page, not what you went looking at.
10. Copying and pasting can also sometimes be detected.
Same idea. Pasting text into a free-response box can contribute to an authenticity check.
That doesn’t automatically mean you cheated, obviously there can be innocent reasons to paste something, but it’s worth knowing that the behavior itself isn’t necessarily invisible.
11. Using AI to answer survey questions is generally not allowed.
Unless the researcher specifically tells you to use ChatGPT, Gemini, Claude, etc. as part of the study, Prolific expects the answer to come from you.
The same basic rule applies to Googling an answer if they’re trying to measure what you already know. If the instructions say you can look something up, that’s different.
12. Correcting your own typos is fine.
Normal Qualtrics doesn’t give the researcher a running transcript of everything you typed and deleted.
If you type something, notice a typo, fix it and continue, they’re normally going to see the final answer.
Same thing if you go back and change an answer because you reconsidered it, assuming the survey allows you to go back.
13. Take free-text questions seriously.
If they ask you to explain your answer and you give them “because I do,” that’s where you can legitimately run into a low-effort rejection.
You don’t need to write more than the question calls for, but if they ask for some detail, give them some detail.
14. Qualtrics can collect more technical information than just your answers.
Depending on how the survey is configured, researchers can potentially get things like your browser, operating system, screen resolution and IP information.
That does not mean they can see your computer, your files, your browser history, or what you’re doing in another tab.
15. Don’t use a VPN, proxy or iCloud Private Relay while doing Prolific studies.
Prolific specifically warns against this because it can interfere with location and account verification. It’s just not worth the potential account problems.
16. Don’t lie about demographics just to qualify for a study.
Your circumstances can change and questions can be worded differently, so a mismatch doesn’t automatically mean somebody lied.
But intentionally changing your age, employment, income, politics, health information, whatever, just because you figured out what answer will get you into the study is obviously a different story.
17. The biggest thing I took away from all of this: don’t try to game being a “good participant.”
Don’t sit around padding your completion time.
Don’t change an honest answer because you’ve given the same answer six times in a row and you’re worried it looks bad.
Don’t add fake variation to a Likert scale.
Don’t rush through repetitive questions either.
Just read what’s actually there, answer honestly, give the amount of effort the question calls for, and submit it when you’re done.
Researchers can see more about our behavior than I realized, especially with Qualtrics, but Prolific also puts some pretty specific limits on what researchers can actually use as a reason to reject somebody.
That’s it. Thank you for coming to my Ted Talk.
