During an external security test, the Gemini model gained access to the systems of three real companies. According to Google, this did not happen as part of an attack on users, and it was not a planned test of those organizations: the error came down to how the test scenario was set up.
How the test broke out of the sandbox
The test was run by Irregular, a company that evaluates how frontier models behave on tasks involving vulnerability discovery. Gemini was supposed to interact with a simulated target, but misconfigured access let it use publicly available data and get into the infrastructure of three real companies.
Google said the affected organizations had been notified and that the model stopped on its own once it realized it had reached a real company rather than the test target. It’s important not to confuse this incident with a campaign run by actual attackers: this was an error during controlled testing. But that is exactly why the story deserves attention — testing systems that have access to tools is becoming just as important as testing the models themselves.
Why this matters for AI agent development
Today’s AI agents can search for information, use a browser, run commands and access connected services. Their usefulness grows with the number of tools they have, which means the cost of a configuration mistake grows too. In this case, the problem was not only the model’s ability to reason, but also the combination of network access, a poorly named test target and insufficient restrictions on the environment.
- A sandbox should be technically cut off from the real network, not just isolated on paper.
- Access rights and keys used in a test should be minimal and temporary.
- The system needs action logs and a way to stop a task quickly.
- Security evaluations should cover not only the model, but also the browser, tools, plugins and data around it.
What changes for everyday users and businesses
Ordinary users don’t need to give up on AI assistants because of one incident. Still, you shouldn’t give an agent more permissions than a specific task requires. For example, an assistant that sorts through documents doesn’t need ongoing access to your work email, cloud storage, messaging apps and browser all at once.
Companies would do well to treat AI tools like a new hire with a restricted access badge: start with a separate test account, don’t connect critical databases at first and regularly review the permissions you have granted. This is especially true for agents that can not only answer questions but also take actions on a user’s behalf.
The bottom line
The Gemini incident shows that safe AI is not a single setting or a single filter. The model, the tools and the environment have to be tested together. The more an agent can do on its own, the more important it is to limit in advance exactly where it can go and what it can change.
Sources and methodology
Analysis of public sourcesChecked 21 September 2026
Leave a Reply