Response to NIST RFI: Security of AI Agent Systems

RESPONSE TO NIST RFI: SECURITY OF AI AGENT SYSTEMS (NIST-2025-0035)

Submitted by: Jason Crittenden, Founder & Research Lead, Shadow Dynamic Systems LLC Date: March 9, 2026

A previous comment submitted to the NIST Cyber AI Profile (January 30, 2026) documented three structural governance failure modes that the current risk profile does not resolve:

That comment documented the problem. This response identifies structural requirements for resolving it.

The structural gap in current AI agent security is not the absence of monitoring, logging, or policy. It is the absence of a governance layer that precedes execution — a control boundary between AI cognition and real-world action that exists as a first-class architectural component.

This response presents Zero Trust Governance (ZTG), a framework that applies the core principle of zero trust architecture (NIST SP 800-207) — eliminating implicit trust — to execution authority.

• Governance and execution collapse into a single temporal plane.
• Reversibility is assumed but not enforced.
• Observability is non-actionable.

The RFI correctly identifies adversarial attacks, backdoors, and misaligned behavior. However, there is a fourth category that is structural rather than adversarial:

Implicit authority delegation — when AI agent systems generate novel action sequences at inference time and those outputs become authority claims that are automatically honored.

This creates vulnerabilities without an adversary:

These failure modes are structural, not adversarial. They arise from architectural decisions about how authority flows from model output to external effect.

• Unauthorized Execution Cascade — stochastic agent executes production mutations within configured scope, producing outcomes no single authorization contemplated.
• Post-Commit Observability Illusion — monitoring detects adverse effects only after irreversible execution.
• Stochastic Policy Delegation — policy evaluation delegated to probabilistic model reasoning.
• Unattributable Authority — irreversible actions execute without traceable binding to human authority.
• Assumed Reversibility — system treats all actions as equivalent regardless of externalization potential.

Multi-agent systems introduce trust transitivity vulnerabilities:

The architectural response: treat inter-agent delegation as a governed effect requiring explicit authorization leases.

• Inter-agent authentication failure — most frameworks do not implement mutual authentication between agents.
• Authority laundering — constrained agent delegates to less-constrained agent, bypassing authorization limits.
• Composition opacity — causal chain from instruction to external effect traverses multiple agents.
• Emergent authority — individual agents operate within scope while combined actions produce unauthorized effects.

STRUCTURAL PREREQUISITES (ZTG-0)

• ZTG-0a Observability — All governance-relevant state transitions must be recorded.
• ZTG-0b Replayability — Governance evaluation must be deterministic.
• ZTG-0c Temporal Integrity — Synchronized time source with bounded skew.
• ZTG-0d Identity Integrity — Cryptographic material traceable to trust root.
• ZTG-0e Governance Consistency — Consistent view of governance state.

• ZTG-1 Mechanistic Boundary — Governance boundaries enforced mechanically, not discretionarily. Authorization must occur before irreversible action.
• ZTG-2 Stasis — When governance invariants cannot be guaranteed, the system halts. Stasis operates at graduated scope: surface freeze → subsystem freeze → system-wide stasis. Containment is proportional to failure.
• ZTG-3 Governed Effect Surface — All agent-generated external effects must occur exclusively through registered, governance-addressable surfaces.
• ZTG-4 Evidence-Coupled Execution — No externally observable effect may exist without simultaneous durable evidence of authorization and execution.
• ZTG-5 Irreversibility of Harm — Every governed action is classified at decision time by the restorability of its harm (Restorable, Mitigable, Irreversible). Irreversible-class actions are subject to stricter controls; Mitigable-class actions record residual harm in provenance. Orthogonal to ZTG-3 system-side reversal strategy.

Five evaluation primitives for any AI agent system:

• Execution Boundary Enforcement — Do all irreversible actions pass through a governance gate prior to execution?
• Deterministic Policy Evaluation — Is authorization logic rule-bound and reproducible?
• Authority Attribution — Do execution rights map to identifiable, accountable actors?
• Replayable Authorization Record — Are governance decisions reconstructible and independently verifiable?
• Commit Verification — Does execution validate governance state before irreversible effect?

The Governed Effect Surface (ZTG-3) provides the primary constraint mechanism. Every interface through which an agent can produce externally observable state change must be registered with declared properties:

Proposal-bound authorization leases bind to the specific proposed action — parameter substitution at execution time invalidates the lease.

• Surface ID, Effect class, Authorization policy
• Reversal strategy (rollback, compensate, none)
• Isolation scope, Audit requirements

These fields developed governance for unbounded agents — any actor with authority to produce irreversible effects:

AI agent systems are the newest class of unbounded agent. The governance requirements are not novel. The failure to apply them is.

• Judicial process — authority is attributable to an identifiable actor, scoped to a specific action, bounded in time and jurisdiction.
• Financial transaction authorization — governance layer distinct from transaction processing layer.
• Clinical practice governance — diagnosis and execution architecturally separated.
• Safety-critical systems engineering — graduated authority escalation, mechanistic safeguards.
• Insurance underwriting — requires bounded loss distributions to price risk.

The security of AI agent systems cannot be addressed through model-level improvements alone. The novel risk is structural: AI agent systems delegate execution authority to stochastic processes without a mechanistic governance boundary between model output and irreversible effect.

Zero Trust Governance applies zero trust security principles to execution authority. Its invariants provide a formal architectural framework for constraining AI agent authority while preserving the benefits of autonomous operation.