[Notes] Agent Security Is a Systems Problem

I recently went over two articles from March-May 2026 that highlight how alignment can improve an agent’s behavior, but security must hold even when the model makes the wrong decision. I found these reads valuable grounding for thinking about agent architectures.

Perplexity’s Security Considerations for Artificial Intelligence Agents develops a layered defense strategy; Christodorescu et al.’s Agent Security Is a Systems Problem treat the model as untrusted and enforce security invariants outside it.

The latter posits the model’s learned judgment should not be part of the TCB, because the model is inherently probabilistic, and we can’t have a probabilistic TCB. A boundary that holds only when the model correctly interprets an instruction or recognizes an attack is a behavioral expectation, not an enforceable restriction.

The papers identify concrete open research problems, including verifiable policy generation: translating a user’s evolving natural-language intent into enforceable constraints. Moving enforcement outside the model is necessary, but letting the same untrusted model freely define its own policy would recreate the original dependency.

Taking the untrusted-model assumption seriously changes how we design agents. A model-generated tool call is a request for authority, not proof of authorization. These requirements become harder when the workflow itself emerges at runtime, rather than following a program whose resource needs are known in advance. There are excellent analogies in here for people working on code integrity, particularly dynamic/JITted code security.

Related designs explore how to enforce these boundaries: aflock places authorization and evidence generation in an external MCP server, while Grimlock uses eBPF-mediated sandbox boundaries and attested channels to enforce identity and constrain delegation outside agent code.