Chatbots often praise instead of criticising, which makes them unreliable for serious thinking and research.
When ideas were framed as coming from someone else, AI gave more honest and balanced feedback.
Experts warn this 'yes-man' behaviour can mislead users, reduce trust, and even encourage unhealthy dependence on AI.
AI chatbots have become immensely popular these days, with people relying on ChatGPT, Gemini, Claude and others just to get their tasks done. However, there is always a negative side. Well, not only me, Yoshua Bengio, one of the world’s most influential artificial intelligence researchers, has raised fresh concerns about how modern AI chatbots are designed to respond to users, arguing that their tendency to be overly agreeable makes them unreliable for serious intellectual work.
SurveyWhile speaking at an episode of The Diary of a CEO podcast released on December 18, Bengio said he stopped using AI chatbots to evaluate his research ideas after noticing a consistent pattern; the systems rarely offered critical feedback and instead responded with praise. As per Bengio, this behaviour limits the usefulness of AI in academic and scientific settings, where honest critique is essential.
Also read: Samsung Galaxy Z Fold 6 price drops by over Rs 62,000: How to grab this deal
In order to test the system, Bengio said that he altered his approach by presenting his own ideas as if they came from someone else. The result, he noted, was noticeably more balanced and critical responses. The experience reinforced his view that many AI systems are optimised to please known users rather than challenge them, a design choice that can distort judgment and reduce trustworthiness.
Also read: Google One rolls out massive price drop on 2TB and AI Pro plans: Everything you need to know
Bengio, a professor at the Université de Montréal and widely regarded as one of the “godfathers of AI” alongside Geoffrey Hinton and Yann LeCun, stated this behavior as a clear example of misalignment in the current AI models. He warned that excessive agreeableness not only undermines factual accuracy but could also encourage unhealthy emotional dependence on AI tools, as users grow accustomed to constant affirmation.
The issue of so-called AI sycophancy has become a growing concern in the technology and research communities. Several studies have found that large language models can reinforce, rather than challenge, questionable decisions. In one widely cited experiment conducted by researchers from Stanford, Carnegie Mellon, and the University of Oxford, chatbots failed to accurately judge problematic human behaviour in a significant number of cases, frequently providing reassuring responses when criticism was warranted.
Also read: Google, OpenAI and xAI sued over alleged use of copyrighted books to train AI
AI companies have acknowledged the issue. Earlier this year, OpenAI reversed an update to ChatGPT after discovering that it caused the system to produce responses that were overly supportive and misleading. Developers across the industry are currently grappling with how to strike a balance between politeness, safety, and honesty in AI-generated feedback.
Follow Us
Ashish Singh
Ashish Singh is the Chief Copy Editor at Digit. He's been wrangling tech jargon since 2020 (Times Internet, Jagran English '22). When not policing commas, he's likely fueling his gadget habit with coffee, strategising his next virtual race, or plotting a road trip to test the latest in-car tech. He speaks fluent Geek. View Full Profile
