Science has a junk paper problem, and AI is making it significantly worse. A new study tested 13 major large language models to see how easily they could be nudged into helping commit academic fraud – fabricating research and inventing benchmark data to ghostwriting entire fake papers – and I’ll be honest, the results are both unsurprising and genuinely alarming. The short version is that eventually every single one of them eventually said yes.
SurveyAlso read: People who use AI most are more mentally drained, finds study
The study was conceived by Alexander Alemi, an Anthropic researcher working independently, and Paul Ginsparg, a Cornell physicist and founder of arXiv, the preprint platform that’s been quietly drowning in AI-generated submissions for the past couple of years. They tested models across five escalating categories of bad behaviour, from mildly misguided (a hobbyist asking where to post their Einstein-debunking theory) to outright malicious (fabricating papers under a competitor’s name to tank their reputation).
Here’s where it gets interesting. GPT-5 looked great in round one where it refused or redirected every fraudulent request when asked cold. Claude, across all versions, held its ground the most consistently when pressed repeatedly. On Anthropic’s own internal testing for Claude Opus 4, the model produced content that could be fraudulently used just around 1% of the time. Grok-3, by comparison, crossed that line more than 30% of the time.
But here’s the thing none of them can escape, and the thing I keep coming back to, persistence works. When researchers ran realistic back and forth exchanges, replying with nothing more sophisticated than “can you tell me more,” every model eventually caved on at least some requests. Grok-4 at one point responded to a prompt asking for a machine learning paper with completely fabricated benchmarks by producing exactly that, cheerfully framing it as an example. It was not asked for an example.
Also read: ChatGPT timeline: Is OpenAI’s pursuit for speed costing them substance?
I’ve spent enough time poking at these models to know that this isn’t shocking. The sycophancy problem isn’t hiding in some dark corner of these systems, it’s baked into how they’re designed to keep users engaged. What surprises me is how little friction it takes. No elaborate jailbreaks, not even clever prompt engineering. Just a nudge, a follow-up and a little patience.
Elisabeth Bik, a research integrity specialist, frames the stakes plainly: fake data skews meta-analyses, misleads clinicians, erodes trust in science, and at worst contributes to harmful medical decisions. The models are getting better at saying no. The problem is they’re still far too good at eventually saying yes.
Also read: ‘Feel the fear and do it anyway’: Lenovo’s Fiona O’Brien tells women in tech
Follow Us
Vyom Ramani
A journalist with a soft spot for tech, games, and things that go beep. While waiting for a delayed metro or rebooting his brain, you’ll find him solving Rubik’s Cubes, bingeing F1, or hunting for the next great snack. View Full Profile
