For the first time, Google says it has found a cyberattack tool it believes was built with AI. The company’s security researchers uncovered a previously unknown flaw in a popular piece of software that criminal hackers were planning to exploit at scale. A patch was issued before any damage occurred. However, Google Threat Intelligence Group’s chief analyst, John Hultquist, told The New York Times: “We believe this is the tip of the iceberg.”
The flaw was found in a popular open-source web administration tool. It allowed attackers to bypass two-factor authentication — the extra verification step many services use to confirm a user’s identity. Crucially, attackers still needed valid login credentials first.
Google’s security team said the exploit was likely written with AI assistance. The code contained detailed explanatory comments and a hallucinated security score — patterns typical of AI-generated output.
The company said it worked with the affected vendor to fix the vulnerability before attackers could use it in a planned mass exploitation campaign.
Google’s broader GTIG AI Threat Tracker report, published today, documents how criminal and state-backed hackers are increasingly using AI tools throughout their operations. Researchers linked to China and North Korea have used AI models to search for software vulnerabilities, sending thousands of automated prompts to analyse known flaws and test potential exploits. Hackers associated with Russia have used AI to generate decoy code — inert programming designed to disguise malicious software.
Analysts also identified an Android malware called PROMPTSPY, which uses Google’s own Gemini AI to navigate a victim’s phone screen and prevent uninstallation. Google said it has disabled accounts linked to the malware. No apps containing it were found on Google Play.
The report also describes hackers building systems to access AI tools at scale while avoiding detection. These include automated account registration pipelines and tools that pool multiple AI accounts to bypass usage limits.
On the supply chain side, a criminal group called TeamPCP claimed responsibility for compromising several popular software repositories in March 2026, stealing cloud credentials from affected build environments. One of the targeted tools was LiteLLM, widely used to connect software to AI services.
Earlier, Anthropic revealed that its Claude Mythos model which remains unreleased for now is able to spot a great number of vulnerabilities in existing global software at scale. Project Glasswing was created with trusted bank and tech partners to use those capabilities for for proactive defence. Meanwhile, Nvidia CEO Jensen Huang warned that China already has the computing power and data center capacity necessary to train an AI model similar to the level of Anthropic’s sophisticated AI model.
