AI agent governance: how to govern agentic AI in the enterprise

NIST describes AI agent systems as systems “capable of planning and taking autonomous actions that impact real-world systems or environments” ^[5]. AI agent governance, also called agentic AI governance, is how an organization keeps those actions within limits it has chosen: who owns each agent, what it is allowed to touch, which actions need a human’s approval, what gets logged, and who can stop it. The European Commission notes that “AI agent” has no legal definition; broadly, an agent receives input from its environment and takes actions that may affect it ^[1].

Enterprise AI governance is the wider program covering every AI system an organization builds, buys or uses: policy, inventory, risk assessment, oversight and accountability. Agent governance is the part of it that deals with autonomy and action. Most organizations should extend their existing AI governance program rather than build a parallel one; the Five Eyes cyber agencies likewise advise folding agentic AI risk into existing cybersecurity frameworks ^[16].

21%

of companies planning to deploy agentic AI report a mature model for agent governance^[21]3,235 leaders in 24 countries, surveyed August–September 2025

Nearly 3 in 4

companies plan to deploy agentic AI within two years^[21]Deloitte survey, August–September 2025

40%

of respondents from organizations with revenue above $1 billion report scaling AI agents, up from 27%^[22]1,719 respondents, surveyed May–June 2026

40%+

of agentic AI projects predicted to be canceled by the end of 2027^[23]Gartner prediction, June 2025

Gartner gives the reasons for its prediction as “escalating costs, unclear business value or inadequate risk controls”, and estimates that only about 130 of the thousands of vendors calling themselves agentic are real ^[23][24]. McKinsey found that about two in ten respondents overall report reaching the scaling phase with AI agents across their organization ^[22]. IBM’s 2026 Cost of a Data Breach study found that more than 20% of organizations reported a breach targeting AI models or applications, and that three quarters say frontier AI threats are prompting them to rethink how agents are deployed ^[26]. Deloitte’s State of AI in the Enterprise research, based on a survey of 3,500 senior leaders, reports that 40% say AI can execute decisions or actions without human approval to at least a moderate degree, and 23% say accountability for a negative outcome remains unclear ^[25].

None of the sources below is an agent-specific law: regulators apply existing AI and cybersecurity rules to agents, and the agent-specific material is guidance. The table summarizes where each source stands as of 30 October 2026; the detail follows.

EU AI Act. The Commission’s AI Act Service Desk says the definitions of an AI system and a general-purpose AI model are enough to cover agents, so the existing rules apply ^[1]. In practice that means: the prohibited practices on harmful manipulation and exploiting vulnerabilities may require safeguards in agent design; Article 50 transparency duties, which apply from 2 August 2026, cover agents that interact with people or generate content; an agent used for a high-risk purpose must meet the Chapter III requirements, from 2 December 2027 for Annex III uses and 2 August 2028 for Annex I products; and autonomy and tool use can be decisive in designating a general-purpose model as having systemic risk ^[1]. Our EU AI Act deadline explainer covers the amended dates.

For a high-risk agent, three provisions map directly onto agent controls. Article 14 requires that the people overseeing a high-risk system can stay aware of automation bias and can intervene or interrupt it “through a ‘stop’ button or a similar procedure” ^[2]. Article 26 requires deployers to assign human oversight to people with “the necessary competence, training and authority”, monitor the system, and keep its automatically generated logs for at least six months ^[3]. Article 73 sets serious-incident reporting deadlines: 15 days in general, two days for widespread infringements and serious incidents under Article 3(49)(b), and 10 days where a person has died ^[4]. Our EU AI Act risk classifier helps decide whether an agent’s purpose is high-risk.

NIST. NIST’s Center for AI Standards and Innovation (CAISI) issued a request for information on securing AI agent systems on 12 January 2026; comments closed on 9 March 2026 ^[5]. On 17 February 2026 it launched the AI Agent Standards Initiative, with three pillars: industry-led agent standards, open-source agent protocols, and research on agent security and identity ^[7]. NIST’s National Cybersecurity Center of Excellence published a concept paper, “Accelerating the Adoption of Software and Artificial Intelligence Agent Identity and Authorization”, open for comment until 2 April 2026 ^[8]. NIST’s summary of the RFI responses (NIST AI 800-5, 18 May 2026) found broad agreement that agents present novel security threats, and that fundamental cybersecurity practices remain relevant but need adapting ^[6]. As of 30 October 2026, CAISI’s guidelines page lists no final agent-security guideline for deployers, only a draft on benchmark evaluations ^[37]. Until one arrives, the NIST AI RMF applies: GOVERN 1.6 asks for mechanisms to inventory AI systems, GOVERN 3.2 for defined roles in human-AI configurations, and MANAGE 2.4 for responsibilities to supersede, disengage or deactivate AI systems ^[18].

In October 2026 NIST, the NSA and CISA published “Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation”, ten voluntary practices grouped into prepare, detect, respond, recover and report ^[9]. It is aimed mainly at organizations training or evaluating highly capable agents on long-horizon tasks, but says it may also apply to deployers of such agents ^[9].

Singapore. IMDA launched its Model AI Governance Framework for Agentic AI at the World Economic Forum in January 2026 and updated it on 20 May 2026 ^[10][11]. Its four dimensions are assessing and bounding the risks, ensuring meaningful human accountability, implementing technical controls and processes, and enabling end-user responsibility ^[11]. It calls for human-approval checkpoints, especially for irreversible actions such as payments ^[36]. The May update adds risks from multi-agent systems, treats the use of third-party agents as raising the likelihood of risk, and adds guidance on automation bias, such as tracking how often people override the system and how quickly they respond ^[11].

ISO/IEC 42001. The AI management system standard doesn’t mention agents, but its controls cover what agent governance needs: A.3.2 AI roles and responsibilities, A.4.4 tooling resources, A.6.2.6 AI system operation and monitoring, A.6.2.8 recording of event logs, A.8.4 communication of incidents, A.9.2 processes for responsible use, and A.10.3 suppliers ^[19]. Our ISO 42001 guide lists every Annex A control.

UK and Five Eyes. The UK National Cyber Security Centre published interim advice on 20 August 2026 and says formal guidance will “build upon, and ultimately supersede” it ^[15]. Its recommendations include running agents in a sandbox with network traffic denied by default, giving every agent “its own unique identity”, scaling oversight to the agent’s autonomy, treating agent activity as user activity in logs, and being able to “pull the plug” immediately ^[15]. Earlier, on 1 May 2026, CISA, the NSA and the cyber agencies of Australia, Canada, New Zealand and the UK published joint guidance on careful adoption of agentic AI services ^[16]. It runs to about 30 pages with more than 100 recommendations, including a verifiable identity for each agent, never granting broad or unrestricted access, human approval for high-impact actions, and independent monitoring that cross-checks agent reports against system logs ^[17].

OWASP’s GenAI Security Project released its Top 10 for Agentic Applications on 9 December 2025, reviewed by the project’s Agentic Security Initiative expert review board ^[12][13]. It lists the ten main agent-specific security risks. The governance column is our mapping to the controls in this guide.

You cannot govern agents you don’t know about. NIST’s GOVERN 1.6 asks for mechanisms to inventory AI systems ^[18], and ISO/IEC 42001 asks for documentation of the tooling resources AI systems use ^[19]. Agents arrive by four routes: built in-house, bought as products, switched on inside software you already use, and run by employees on their own initiative. Our shadow AI guide covers discovering the last two. For each agent, record:

• A named owner who is accountable for it; NIST’s GOVERN 2.1 asks for documented roles and responsibilities for AI risks ^[18].
• Purpose and scope: the tasks it may perform and those it may not.
• Models and providers, including versions.
• Autonomy tier and the actions that need human approval (see below).
• Tools, MCP servers, skills and other agents it can call ^[14].
• Identity and credentials: the agent’s own identity, what each credential grants and when it expires ^[15].
• Data it can read and write, and its sensitivity.
• Legal classification, for example whether its purpose is high-risk under the EU AI Act ^[1].
• Where its logs go and who reviews them, and who can halt it and how ^[15].
• Next review date.

Our free AI risk register can hold the inventory to start. At scale, discovery tools find agents automatically; examples are in the vendor section below.

Give every agent its own identity. The NCSC says all agents should be assigned their own unique identity, with credentials limited to the task and the shortest practical lifetimes ^[15]; the Five Eyes guidance says each agent should have its own secure, verifiable identity and temporary credentials for sensitive actions ^[17]. Agents should not borrow a human’s account or a shared service account, because then nobody can tell from the logs who did what.

Grant the least privilege the task needs. The NCSC recommends giving an agent “only the permissions it needs for the task being performed” ^[15]. Scope permissions to operations, not just systems: reading a record should not imply the right to delete it ^[31]. The Five Eyes guidance warns against ever granting agents broad or unrestricted access, especially to sensitive data or critical systems ^[17].

Set human approval by action, not by agent. The NCSC describes three models (human in the loop, human on the loop and human out of the loop) and recommends human oversight alongside technically enforced controls for higher-risk scenarios ^[15]. Singapore’s framework puts approval checkpoints on irreversible actions such as payments ^[36], and the Five Eyes guidance asks for human review or approval of high-impact actions ^[17]. The tiers below are our suggested starting point; adjust them to your risk appetite.

The Model Context Protocol (MCP) is the common way agents connect to tools and data. Each MCP server an agent can reach is effectively a set of permissions plus a piece of third-party software, so it belongs in the inventory and under change control. OWASP lists agentic supply chain vulnerabilities and tool misuse among the top agent risks ^[14], and our shadow AI guide describes a fake MCP server that copied every outgoing email to an attacker.

• Keep an allowlist of approved MCP servers and tools, and block the rest. Products now enforce one organization-wide list across agents, IDEs and agentic apps ^[28][29].
• Discover unapproved servers. Inventory tools detect MCP servers and agent processes alongside models and applications ^[34][30].
• Scope each tool to the operations needed. Access to a tool should not mean access to every operation it supports ^[31].
• Give connectors their own short-lived credentials. The NCSC recommends limiting an agent’s credentials to what the task requires, with the shortest practical lifetimes ^[15].
• Route traffic through one enforcement point where you can. AI gateways now sit inline between agents and models, MCP servers and other agents, applying approved-tool policies and short-lived agent identities ^[33].
• Review changes. Treat a new MCP server or a new version like any new third-party software, under ISO/IEC 42001’s supplier control ^[19].

The NCSC says agentic AI activity should be treated as a form of user activity, with logs protected from tampering and, where possible, immutable ^[15]. The October 2026 NIST, NSA and CISA practices call for tamper-resistant logs with reliable timestamps sufficient to reconstruct agent trajectories ^[9], and the Five Eyes guidance recommends independent monitoring that cross-checks what agents report against system logs ^[17]. For high-risk systems under the EU AI Act, deployers must keep automatically generated logs for at least six months ^[3]; ISO/IEC 42001 has a control for recording event logs (A.6.2.8) ^[19].

• Who: the agent’s identity and the human or system that started the task. Some products link agent and agent-to-agent activity back to the person who started the workflow ^[29].
• What it was asked: instructions and the context it retrieved, with sensitive data masked.
• What it did: every tool call with its arguments and result, and any data read or written.
• What was decided: approvals and refusals, and policy blocks with the rule that fired ^[29].
• Which versions: model, prompt, tools and policy.

Because the NCSC treats agent activity as a form of user activity ^[15], send agent logs to the security operations team rather than leaving them in a developer dashboard. For tracing and evaluation tools, see our LLM observability ranking.

1. 01

   Prepare

   Name who can halt each agent, rehearse it, and write agent scenarios into the incident plan. NIST’s AI RMF asks for mechanisms and assigned responsibilities to supersede, disengage or deactivate AI systems (MANAGE 2.4) ^[18].

2. 02

   Detect

   Monitor agent behavior, not just availability. The NIST, NSA and CISA practices include validating and securing the monitors themselves ^[9].

3. 03

   Halt

   Be able to stop an agent immediately. The NCSC says that can mean restricting network access and cutting the agent’s link to its model inference infrastructure, not only stopping its processes ^[15]; the NIST practices call for automatically shutting down affected agent runs on detection of a control failure ^[9].

4. 04

   Preserve evidence

   Keep tamper-resistant logs that reconstruct what the agent did ^[9], and revoke the credentials it held.

5. 05

   Recover

   Verify and strengthen containment before letting the agent run again ^[9].

6. 06

   Report

   Communicate incidents to the people affected (NIST MANAGE 4.3; ISO/IEC 42001 A.8.4) ^[18][19]. For high-risk systems under the EU AI Act, serious incidents must be reported to market surveillance authorities within 15 days, or two or 10 days in the cases Article 73 sets out ^[4]. Our EU AI Act incident report builder drafts the report.

An AI governance committee is the cross-functional group that owns AI policy, approves higher-risk AI uses and agents, and answers to senior leadership for AI risk. NIST’s AI RMF expects executive leadership to take responsibility for decisions about AI risks (GOVERN 2.3) and decisions to be informed by a diverse team (GOVERN 3.1) ^[18]. The IAPP found that 77% of organizations are already working on AI governance ^[27]. For agents, the committee is where autonomy tiers, approval thresholds and the right to halt an agent get decided, so they are not left to each development team.

A public-sector model. OMB memo M-25-21 (3 April 2025) requires each US CFO Act agency to convene an AI governance board chaired at deputy secretary level, with the agency’s Chief AI Officer as vice-chair, and with representation from offices responsible for IT, cybersecurity, data, budget, statistics, legal counsel, privacy, civil rights and civil liberties ^[20]. Agencies may use existing governance bodies ^[20]. The same shape works in companies: a senior executive chair, a governance lead who runs it, and every function that carries AI risk.

What the charter should say. Keep it to two or three pages:

1. Purpose and scope: all AI systems the organization builds, buys or uses, including agents run by staff and AI features in existing software.
2. Authority: approve, condition or reject AI uses above a set risk tier; set autonomy tiers and approval thresholds; order an agent halted; require remediation.
3. Decision rights: which decisions the committee makes, which it delegates to the governance lead or business owners, and which go to the board.
4. Membership and quorum: named seats, deputies, and who must be present for a high-risk approval.
5. Inputs: the intake form, the inventory, risk assessments, test results and incident reports.
6. Outputs: decisions with reasons, an updated register, and a regular report to senior leadership.
7. Escalation: how urgent issues reach the chair between meetings.
8. Review: when the charter itself is reviewed.

Cadence. None of the frameworks above sets a meeting frequency; OMB’s memo doesn’t either ^[20]. A common pattern, and our suggestion, is a monthly working meeting for intake and approvals, a quarterly risk review for senior leadership or the board, an ad hoc meeting within a day or two of a serious AI incident, and an annual review of the charter and AI policy. To avoid becoming a bottleneck, approve classes of agents by autonomy tier and delegate tier 1 and 2 decisions, keeping the committee’s time for tier 3 agents and exceptions.

Our AI policy generator drafts the policy the committee owns, and the AI readiness assessment shows where your program stands.

1. 01

   Assign ownership

   Charter the AI governance committee and name an executive who is accountable for AI risk (see above) ^[18][20].

2. 02

   Inventory every agent

   Built, bought, embedded and employee-run, with a named owner, tools, credentials and data for each ^[18][15]. Start with the AI risk register.

3. 03

   Classify

   Assign each agent an autonomy tier and check whether its purpose is high-risk under the EU AI Act or another law ^[1]. Use the EU AI Act risk classifier.

4. 04

   Write the rules

   Extend your AI policy to cover agents: who may build or run them, approval thresholds, prohibited actions and MCP rules. The AI policy generator is a starting point.

5. 05

   Give each agent an identity

   One identity per agent, least-privilege permissions scoped to operations, and short-lived credentials ^[15][17].

6. 06

   Set approval thresholds

   Human in the loop for high-impact and irreversible actions, human on the loop for bounded ones, and technical blocks for prohibited actions ^[15][36].

7. 07

   Control tools and MCP

   Allowlist MCP servers and tools, discover unapproved ones, and route traffic through a gateway where possible ^[28][33][34].

8. 08

   Test before release

   Test against the OWASP agentic risks, including goal hijacking and tool misuse ^[14]. Our AI red team tools ranking covers testing tools, and the AI guardrails guide covers runtime controls.

9. 09

   Log, monitor and rehearse the halt

   Immutable logs in security operations, behavior monitoring, and a tested way to stop each agent immediately ^[15][9].

10. 10

    Review and train

    Re-check each agent’s autonomy and permissions on a schedule, track approval override rates ^[11], and train approvers on automation bias ^[2].

We included directory vendors whose own product pages, checked on 30 October 2026, document at least three of: agent discovery, agent identity or permissions, MCP or tool control, runtime enforcement, and audit trails. We did not run hands-on tests, and no vendor paid to be listed. For a full ranking of agent security products, see AI agent security tools; for inventory and policy workflow platforms, see AI governance platforms.

1. 01

   Noma Security

   Best for: Per-agent permissions and approvals

   Builds a live registry of agents, MCP servers and skills and sets a baseline of which are allowed; defines which actions an agent may take based on its owner’s identity, scopes tool access to operations, and can route a blocked action to a human for approval with an audit trail. It says it can stop destructive actions, alert a human or trigger a kill switch.^[30][31][35]

   Noma Security profile →

2. 02

   WitnessAI

   Best for: Tying agent activity to the people behind it

   Discovers agents and the MCP servers and tools they reach, enforces one organization-wide list of approved MCP servers and tools for agent traffic that passes through it, and records the user, agent, tool and rule for each blocked call. Its pages do not describe a human-approval workflow for agent actions.^[28][29]

   WitnessAI profile →

3. 03

   Prisma AIRS

   Best for: Palo Alto Networks customers who want one gateway for agent traffic

   Palo Alto Networks’ platform covers discovery of AI apps and agents, agent security and agent identity controls. Its AI Gateway, generally available since July 2026, is a single enforcement point for LLM, MCP and agent-to-agent traffic and gives agents a verifiable, short-lived identity at run time.^[32][33]

   Prisma AIRS profile →

4. 04

   Cisco AI Defense

   Best for: Cisco customers who need to find agents and MCP servers in the cloud

   Its AI inventory detects MCP servers and agent processes alongside models and applications, it scans MCP servers for compromised or malicious assets, and its agent behavior protection detects and blocks unsafe actions such as unauthorized tool use and privilege escalation.^[34]

   Cisco AI Defense profile →

• AI agent security tools →The products that discover, control and protect AI agents, ranked.
• Shadow AI →Find the AI tools and agents staff use without approval.
• AI guardrails →Runtime controls on prompts, outputs and agent tool calls.
• AI policy generator →Draft the AI policy your committee will own.
• AI readiness assessment →Score where your AI governance program stands today.
• ISO 42001 guide →The AI management system standard and its Annex A controls.