ChatGPT, Gemini, Claude, Meta AI easily made phishing emails to scam elders, in a study

In an exclusive investigation, Reuters revealed how some of the world’s most widely used AI chatbots – including ChatGPT, Gemini, Claude, Meta AI, Grok, and DeepSeek – can be manipulated into creating persuasive phishing content targeting elderly internet users. The findings highlight a growing concern that, despite promises of robust safeguards, generative AI is already being exploited in ways that could put vulnerable populations at greater risk of fraud.

Survey

Reuters, working with Harvard researcher Fred Heiding, put the chatbots through a series of tests designed to mimic how a cybercriminal might try to use them. On the surface, most of the systems initially declined to generate harmful content when directly asked to craft phishing emails. But with slightly altered wording, many quickly complied. Some went beyond writing the emails themselves, offering campaign strategies, suggesting domain names, and even advising on how to keep victims unaware they had been defrauded for as long as possible.

Add As A Trusted Source For Google.

Add as a preferred source on Google

Also read: Are AI chatbots safe for children? Big tech companies need to answer, says US FTC

My own experiment

Curious to see how these safety nets hold up, I decided to try the exercise myself. The differences between the chatbots were striking. Grok, developed by Elon Musk’s xAI, was the least resistant. Without much effort or elaborate persuasion, it drafted a phishing-style email almost immediately.

Gemini, Google’s flagship chatbot, proved harder to bend. It resisted outright attempts to draft a fraudulent email but eventually provided a different kind of assistance. Instead of producing a full phishing message, it offered breakdowns: lists of potential subject lines, outlines of what the body of the email should contain, and explanations of how scammers typically frame urgent messages. In other words, it handed me all the building blocks of a scam without stitching them together.

ChatGPT followed a similar pattern. It refused to generate an email directly but supplied categorized examples of phishing tactics, from subject-line structures to commonly used persuasive phrases. To someone with malicious intent, these suggestions could be enough to assemble a convincing scam in minutes.

This mirrored the Reuters findings: the guardrails in place are inconsistent and often porous. What one chatbot refuses to do outright, another may achieve indirectly, sometimes even in the same session with only slight rephrasing of the request.

A dangerous inconsistency

The Reuters investigation underscored this inconsistency in striking ways. In one test, DeepSeek, a Chinese-developed model, not only produced scam content but also suggested delaying tactics designed to stop victims from catching on quickly. The same chatbot, in another session, would refuse the identical request. This kind of unpredictability makes the technology difficult to regulate and even harder to trust when it comes to safety.

Also read: What if we could catch AI misbehaving before it acts? Chain of Thought monitoring explained

Such inconsistencies may not seem dramatic at first, but they lower the barrier to entry for would-be scammers. A determined user does not need specialized technical skills or deep knowledge of social engineering. They only need persistence and access to a chatbot that occasionally slips past its own restrictions.

Real-world impact 

The investigation went further than simulated prompts. To measure effectiveness, Reuters and its academic partner designed a controlled trial involving 108 senior citizen volunteers. These participants consented to receive simulated phishing messages based on AI-generated content. The results were sobering: around 11% clicked on at least one fraudulent link.

For older adults, who are disproportionately targeted by cybercrime, this statistic is alarming. Even a modest success rate can translate into enormous financial and emotional damage when scams are launched at scale. With AI making it faster and cheaper to produce convincing emails, the danger for seniors, many of whom are less familiar with digital deception grows exponentially.

Industry reactions

Confronted with the findings, AI companies acknowledged the risks but defended their efforts. Google said it had retrained Gemini in response to the experiment. OpenAI, Anthropic, and Meta pointed to their safety policies and ongoing improvements aimed at preventing harmful use. Still, the investigation shows that these measures remain patchy. The difference between refusal and compliance often depends on subtle changes in phrasing or persistence, loopholes that malicious actors are adept at exploiting.

Generative AI has already transformed creativity, productivity, and online communication. But this investigation demonstrates its darker potential: the ability to industrialize fraud. Where a scammer once needed fluency in English and skill at crafting persuasive language, they now need little more than time and a chatbot account.

For regulators and industry leaders, the challenge is balancing innovation with accountability. Policymakers are already debating how best to oversee these tools, but the Reuters study makes clear that urgency is mounting. For ordinary users, especially the elderly, the best defense remains awareness and education. Spotting red flags, questioning urgent requests, and hesitating before clicking on links are habits that matter now more than ever.

The investigation, coupled with hands-on tests, paints a sobering picture. AI’s promise is immense, but so is its potential for harm. Unless safeguards improve quickly, scammers may find themselves with powerful new partners in crime, partners designed to be helpful, but not yet reliably safe.

Also read: OpenAI’s o3 model bypasses shutdown command, highlighting AI safety challenges

Follow Us

Vyom Ramani

A journalist with a soft spot for tech, games, and things that go beep. While waiting for a delayed metro or rebooting his brain, you’ll find him solving Rubik’s Cubes, bingeing F1, or hunting for the next great snack. View Full Profile