Why AI Agents Are Unintentionally Bad Actors Inside Your Infrastructure

We had a whole industry pop up for firewalls and security software to protect against intentionally bad actors trying to compromise your infrastructure. Now AI agents are doing the same damage from the inside — except they are not trying to. One engineer burns through an entire Anthropic budget in seven hours. Zombie agents run unmonitored after their creators leave the company. An autonomous coding agent gets loose in a production database. None of it is malicious. All of it is expensive.

On a recent episode with Demetrios, Guild.ai CEO James Everingham made the case that agents need to be governed the same way employees are — not because they are replacing people, but because they share the one characteristic that makes people hard to manage at scale: they are non-deterministic. Ask the same agent the same question twice and you will get different answers. Just like a human. And just like humans, that means you need guardrails, policies, and accountability structures around them.

The conversation covered everything from why mandating AI adoption backfires, to how Meta eliminated code freezes with an agent-driven risk scoring system, to why source control itself needs to be reinvented for a world where humans and AI co-author code at character level.

Everingham spent the early part of his career designing operating systems and programming languages at Borland, so when he talks about agent governance, he reaches for a metaphor most infrastructure people understand immediately.

"If you think about the way an operating system is designed, you have this kernel, and it goes through different layers where you have your device drivers and your applications, and you basically set APIs that sort of dictate if you can reach in or reach out. And if you reach in, it's very dangerous."

The argument — and the core idea behind what Guild calls the AI agent control plane — is that a company deploying agents needs the same layered architecture. There is a kernel — your critical infrastructure, your production data, your financial systems — and there are applications running on top of it. The control plane sits in between, enforcing which agents can reach in and which ones are sandboxed at the application layer.

Without that layer, companies default to what Everingham calls the "Wild West" pattern: engineers run agents on their developer laptops with unbridled access to infrastructure, nobody knows what is being accessed or how much it costs, and the first sign of a problem is usually a bill or an outage. It is the same dynamic behind agent sprawl — the problem is not that agents exist, it is that nobody is governing them.

The most practical insight from the conversation was that agent governance is not a single toggle. Different teams inside an enterprise need fundamentally different controls over the same agents, and they need to configure those controls independently.

"The finance team might be the ones signing off on your third-party model budgets. They might want to get in and set certain budgets or certain circuit breakers around agents, teams, people. The security team might have some different thing where they want to sandbox this off and don't want this agent to be able to access these very critical systems."

This creates a problem that most governance proposals gloss over. It is not enough to say "agents need guardrails." You need to answer: whose guardrails? A compliance officer at a regulated financial institution cares about whether an AI agent is even allowed to perform a code review. A security lead cares about credential scoping and audit trails. A finance director cares about token budgets per team. Those are three different people with three different risk models configuring three different policy layers on the same agent.

Everingham sees this multi-stakeholder governance as one of the hardest problems in the space — and one of the reasons you cannot just vibe code your own governance layer and expect it to hold up.

The cost problem came up more than once, and Everingham did not mince words.

"We had one partner we were working with that came to us and said, 'You know, we had one engineer that went through our entire Anthropic budget in seven hours on their laptop, and no one knew.'"

The issue is not individual overspending. It is organizational blindness. As Tim Osborn detailed in The State of Agent Governance, companies like Uber and ServiceNow have blown through entire annual token budgets in months. Most companies deploying agents today have no visibility into which agents are consuming tokens, which teams are driving costs, or where the budget is actually going. The first time leadership finds out there is a cost problem is when the invoice arrives.

Token-level budget controls — per agent, per team, per model — are not a nice-to-have. They are table stakes for running agents in any organization past the experimentation phase.

The conversation took a sharp turn into developer culture when Demetrios asked about the gap between AI tool adoption and actual productivity gains. Everingham, who built the developer platform at Meta, had a specific take: mandating usage is the wrong move.

"The mistake that I see is that a lot of leaders are mandating AI tool usage. You want to earn the usage. You want the tools to earn the usage by demonstrating value."

He drew a parallel to the early IDE days at Borland in the late 1980s. Turbo Pascal pulled the compiler, debugger, and editor into a single integrated environment. Senior engineers dismissed it as a toy — real engineers used the command line. But over time, the IDE earned adoption by proving developers could be more effective with it.

The same dynamic is playing out with AI tools today. Engineers who have spent years building muscle memory with their own workflows are not going to throw them away because a manager tells them to. The solution at Meta was to put challenges in front of teams that could only be solved using the new tools.

One of those challenges produced something they eventually open-sourced: Diff Risk Score. The team built agents that sat between CI/CD and source control, pulled diffs, and used AI to measure the risk of each change based on the engineer, the area of the codebase, and the nature of the change. Low-risk changes could ship. The result: they eliminated the long-standing December-through-January code freeze that had plagued the company for years.

Another team built an onboarding agent for the Instagram codebase. An engineer could say "I want to work on this camera filter" and the agent would respond with the relevant files, system diagrams, and history — cutting the ramp-up time from a week to hours.

"As soon as one engineer did some very high-impact thing and it was published, the other engineers were like, 'Okay, I can see how I can apply that,' and they could build on it."

The takeaway is not that you need a specific tool. It is that making agent work visible — through a centralized hub where engineers can see what agents exist, what they do, and fork them — is what drives organic adoption. Mandates drive resentment.

Everingham pushed back hard on the common framing of agents as single autonomous units that own entire workflows end to end. His view is that agents should be thought of as capabilities, and engineering is evolving toward capability architecture.

"If you built one agent that did all of that, that would be pretty hard to even troubleshoot. If something goes wrong, how do you even find out if it just goes into a black box and comes out?"

At Guild, they run eight agents that work together in a chain. One monitors server logs hourly. If it finds an error, it creates an issue, which triggers a planning agent. The planning agent produces a remediation plan, which feeds into a coding agent that writes a fix and a testing agent that builds a test for it.

Each agent is specialized. Each can use a different model — frontier models for coding, smaller models for simpler tasks. Each can be debugged, replaced, or forked independently. That composability is the point.

The analogy he keeps coming back to is mainframes versus microservices. For decades, the answer to scale was "build a bigger computer." Then the industry realized that racks of smaller, specialized servers were a better architecture — predictable scaling, easier troubleshooting, more resilient. Agents are following the same arc.

"Agents are more like Legos than complete machines, and engineering is gonna be putting those Legos together well."

One of the more forward-looking threads in the conversation was about the infrastructure that breaks when agents start writing code alongside humans at scale.

Demetrios mentioned that Lovable is creating so many repos per second that it maxes out GitHub's rate limits. Everingham was not surprised. At Meta, no existing source control system could handle the monorepo — billions of lines of code required custom file systems that loaded on-demand because the codebase literally would not fit on a hard drive.

That was with humans writing code at human speed. With agents now writing significantly more code, version control systems are going to hit their limits much faster.

But the deeper problem is not throughput. It is provenance.

"You have a combination of humans and AI authoring code together. When that PR gets submitted, how do you know what code was written by a human and what was written by an LLM? You need character-level tracking."

Line-level diffs — the standard in every version control system today — are not granular enough for a world where an AI might have written three characters in the middle of a line that a human wrote. You need to know which characters came from where, which model produced them, and how that maps back to evaluations. Without that, you cannot measure whether your agents are actually improving, and you cannot satisfy compliance requirements in regulated environments.

The conversation closed on what Everingham sees as the biggest unsolved problem: how do you safely transfer context between agents, especially across organizational boundaries?

"I'm not worried about how I would transfer context from one agent to an agent in my own company. I'm worried about if I pull in infrastructure or agents from another company, how do I transfer context to them so they can do their job well?"

MCP helped standardize how agents access tools. But Everingham thinks the industry needs something richer — a structured format for transferring context that includes metadata about what the context contains, where it came from, and how it should be used. Think customer data in one field, business data in another, call history in a third, with clear agreements on the structure so that another system can translate it into something usable.

He is not prescribing a solution. He is naming the problem: agents today operate with ad-hoc context that disappears when the runtime disappears. Long-running sessions need context that persists. Cross-company agent interactions need context that transfers. None of that has been standardized yet.

"We just need to agree on what the standard is. You saw what MCP did — once the standard was out there, suddenly a lot more tools were useful to AI. But it's not gonna stop there."