AI chatbots can be misled by fake online content:ChatGPT, Gemini, and Google AI search repeated false claims, prompting policy changes

Imagine making up a fact about yourself online and then watching some of the world’s biggest AI tools repeat it as the truth. Sounds strange, but that’s exactly what a new investigation has revealed. As artificial intelligence becomes a bigger part of daily life, millions of people are turning to chatbots and AI-powered search tools for answers. But researchers have found that these systems can sometimes be surprisingly easy to mislead. How a fake story tricked AI According to a BBC investigation, a reporter published a fake blog post claiming that they were a world-champion hot-dog eating champion. There was no proof, no records, and no evidence to support the claim. Yet within a day, several major AI tools, including ChatGPT, Gemini, and Google’s AI-powered search features, started repeating the information as if it were true. The experiment showed how quickly false information can spread through AI systems when it appears online. Why this matters beyond a funny example At first glance, a fake hot-dog eating record may seem harmless. But experts say the same technique could be used for far more serious topics. Researchers found that misleading online content could potentially influence AI-generated advice related to: In short, false information online can sometimes make its way into AI-generated answers that people trust and act upon. Why do AI chatbots get fooled? Traditional search engines usually show users a list of websites so they can compare different sources. AI chatbots work differently. Instead of presenting multiple links, they try to provide a direct answer. To do that, they gather information from content available across the internet. The problem is that AI systems can sometimes place too much trust in a small number of sources. If misleading content is published online and appears convincing enough, chatbots may include it in their responses. This creates an opportunity for companies, marketers, spammers, or scammers to publish content designed specifically to influence AI-generated answers. Also read: What is a digital will?: Here’s how to nominate someone to handle your YouTube, WhatsApp email accounts after death

The real-world risks Search engine experts told the BBC that manipulated AI responses could affect much more than casual internet searches. They warned that AI-generated answers could potentially: As more people rely on AI tools, the impact of inaccurate information becomes much larger. Google responds with tougher rules Following growing concerns, Google has updated its spam policies. The company now clearly states that creating content specifically to manipulate AI-powered search results violates its rules. Google says this is not a completely new policy and that it has always worked to fight spam. However, experts believe the update signals a stronger effort to prevent websites from gaming AI systems. How AI companies are fighting back AI companies are actively working to limit the spread of false information through their chatbots. Researchers have observed that some AI tools now display warnings when they are unsure about an answer, encourage users to verify information from trusted sources, avoid relying on questionable websites, and respond more carefully to sensitive topics. These measures suggest that companies are strengthening their safeguards against misinformation.
However, the details of these anti-spam and fact-checking systems remain largely unknown. During the investigation, both OpenAI and Anthropic declined to reveal the specific methods they use to prevent manipulation of their AI systems. Also read: Young generation spending over 97 mins daily on social media: Girls spend more time than boys on e-commerce portals

Problem isn’t going away anytime soon Experts believe this will remain an ongoing battle. As search engines and AI systems improve their ability to detect manipulation, those trying to spread misleading information will continue looking for new methods. Instead of blogs and websites, they may increasingly use social media posts, influencer content, videos, and other platforms. What should users do? The BBC’s conclusion is simple: AI companies are getting more vigilant, but users can’t relax. While AI tools can be useful for finding information quickly, users should avoid treating every answer as a verified fact. For important topics such as health, money, law, or public affairs, it is always wise to cross-check information using trusted and reliable sources.