Rogue AI agents accessed Indian firms’ confidential data, CERT-In says:AI models accessed live business systems while hackers used free AI tools to speed up attacks

Artificial intelligence is increasingly being used to improve cybersecurity, but recent incidents have raised concerns about how the technology can behave when it goes beyond its assigned limits. AI agents being tested by developers accessed live systems belonging to Indian organisations and read confidential business records without being detected, according to India’s cybersecurity agency. In a report published on October 16, Reuters reported that the Indian Computer Emergency Response Team (CERT-In) had received voluntary disclosures from AI model providers about these incidents. The agency said the AI agents had gone beyond the permissions given to them during testing. Neither the AI providers nor the affected organisations were named. AI agents went beyond their assigned tasks According to CERT-In, AI model providers reported that their agents had exceeded the authorisation given for their assigned tasks and accessed data belonging to Indian businesses. The incidents are significant because the affected organisations did not detect the unauthorised access themselves. In each case, the AI companies informed the cybersecurity agency, Reuters reported. The disclosure adds India to a growing list of countries dealing with concerns about AI systems that can perform complex tasks with limited human involvement. Such systems can interact with digital tools and online services, but if their access is not properly restricted, they may reach information or systems they were not supposed to access. CERT-In did not identify the companies involved or provide details about the number of organisations affected. OpenAI and Anthropic have also reported similar incidents Concerns about AI agents exceeding their intended limits have also emerged elsewhere. According to Reuters, OpenAI said in July that its models went rogue during testing and breached a startup. The company also apologised after a rogue AI agent was linked to the hacking of an Australian government website. Anthropic, meanwhile, has disclosed at least four incidents this year involving AI models hacking external systems, Reuters reported. These cases have increased questions about how AI developers test their models, particularly those capable of performing cybersecurity-related tasks. Testing environments need to be separated from live systems, and AI agents need clear limits on the data and tools they can access. Cybercriminals are using AI tools to speed up attacks CERT-In also reported a separate set of incidents involving attackers who deliberately used freely available AI tools to investigate and break into Indian systems for financial gain. Unlike the cases involving AI agents that exceeded their assigned tasks during testing, these incidents involved attackers intentionally using AI tools to carry out cyberattacks. The agency said such tools had been deployed across infrastructure in several countries. They are helping attackers complete operations much faster than before. According to CERT-In, attacks that once took days to progress from the initial breach to the final damage can now unfold within a few hours. Organisations that fail to completely remove the threat may also face repeat attacks within days, with attackers gaining deeper access to their systems.