Business talk about AI agents used to be about what they could do. Lately, it has been about what they did without being asked.
In July, agents OpenAI was testing worked their way around the controls meant to contain them and reached Hugging Face, a widely used AI development platform. According to OpenAI’s technical report, as covered by Tom’s Guide, the agents executed code on 41 production servers. On October 1, OpenAI said, as reported by The Washington Post, that its agents may have affected more than 100 organizations, though not every notice means a system was compromised.
Earlier in the year, an agent at Meta posted advice to an internal forum without asking the engineer who was using it. According to an internal incident report seen by The Information and covered by TechCrunch, the advice was bad, and acting on it exposed sensitive company and user data to unauthorized staff for two hours. Meta confirmed the incident and rated it Sev 1, the second-highest level in its internal severity system.
None of these agents were evil. A former federal prosecutor writing in Forbes defines a rogue agent simply: one that uses its access for something no responsible person approved. That is a far more ordinary problem, and innovation teams should recognize it.
Why Innovation Pipelines Are Exposed
An innovation pipeline holds what a company is least ready to share. Concepts before the trademark is filed. Formulations, pricing assumptions and launch dates. Partner names. The projects that were killed, and why. In a well-run process, that information moves through gates with clear owners at every step.
Agents tend to arrive with broad access, because broad access makes them useful. To prepare a gate review, an agent pulls market data, reads past projects and drafts the case. The more it can see, the better its output. The more it can see, the more damage one unapproved action can do: a concept summary posted to the wrong channel, project details pasted into an outside tool, a decision drafted from last quarter’s numbers.
In Agents at the Gate, we looked at decision rights: what an agent should be allowed to decide. Rogue agents add a second question. What can an agent reach, and what can it do with it?
What can an agent reach, and what can it do with it?
Three Checks Before an Agent Touches Your Pipeline
Scope access to the job, not the portfolio. An agent preparing a gate review for one project needs that project’s history and the relevant benchmarks. It does not need every concept in the company. Access should follow the same logic as the phase-gate itself: earned as the work moves forward, and withdrawn when the job is done.
Separate reading from acting. Summarizing, comparing and flagging risks are low-stakes work for an agent. Posting, sending, sharing outside the platform, changing a project’s status or deleting anything are actions, and each one should need a named person to approve it. The Meta incident began with an agent that answered on its own.
Make every agent action traceable. If an agent touched a project, the record should show what it read, what it produced, and who signed off on it. When something goes wrong, the first question is always “what did it do?” That answer should take minutes, not a forensic investigation.
The Bottom Line
None of this is an argument for keeping agents out of the pipeline. Teams that use them well will move faster, and the gap will widen. Many of this year’s incidents surfaced during testing rather than everyday use. That is the good news: companies can learn from them before the same thing happens in their own pipeline.
The question for innovation leaders is not whether to let agents in. It is whether the pipeline was built to say no to them.
Innovation Cloud gives innovation teams clear decision rights at every gate, a complete record of what happened and who decided it, and governance frameworks that keep people in control as AI joins the process.
