The perimeter you built for the last decade assumed inputs came from users and instructions came from you. Enterprise AI agents dissolved that assumption. Every email an agent reads, every SharePoint page it retrieves, every ticket it summarizes, and every tool it calls is now a potential instruction channel. Prompt injection attacks on enterprise AI agents do not exploit a bug in the model. They exploit the architecture of how agents work.

Two production incidents in 2025 removed any remaining doubt that this is theoretical. EchoLeak, disclosed by Aim Security and tracked as CVE-2025-32711 with a CVSS score of 9.3, allowed a single crafted email to make Microsoft 365 Copilot exfiltrate internal data with no user interaction. The attack chained four bypasses, including Microsoft's own cross-prompt-injection classifier and link redaction. Weeks later, ForcedLeak (CVSS 9.4) demonstrated the same pattern in Salesforce Agentforce: a payload hidden inside a Web-to-Lead description field convinced the agent to leak CRM records to an attacker-controlled domain that had passed every allowlist check.

The uncomfortable read for CISOs is that both attacks succeeded against systems that were, by every conventional measure, secured. Identity was in place. DLP was configured. The vendor's built-in guardrails were on. And the data still walked out.

TL;DR

Prompt injection is not a model bug that vendors will patch. It is the structural consequence of building agents that read untrusted content and take autonomous action. Defending enterprise AI agents on the Microsoft stack requires a new trust-boundary model, a sequenced activation of controls you already own, and adversarial testing before production, not after an incident.

Key Takeaways

• Agent security is a distinct discipline. Governance policies, traditional DLP, and identity controls do not see agent-specific threats.
• Indirect prompt injection through grounded content is now the dominant attack class. Every data source an agent reads is a new trust boundary.
• The Four-Zone Agent Trust Model gives executives a mental map: user prompt, grounded content, tool outputs, and generated actions each require their own controls.
• Copilot Studio is secure by default within a bounded scope. Grounding-source hardening, tool allow-listing, and output DLP remain the customer's responsibility.
• Continuous red-teaming with tools like Microsoft's PyRIT is a pre-production gate, not a post-launch nice-to-have.

The Trust Boundary Has Moved

Traditional application security assumed a clear line between instructions (from developers) and data (from users). Input validation, WAFs, and DLP were all designed around that boundary. AI agents erase it. When a Copilot reads a SharePoint document to answer a question, that document's contents become part of the model's operating context. If an attacker put instructions inside the document, the agent may follow them.

Microsoft's own guidance names this shift plainly: the AI's inability to distinguish between user input and external content makes traditional input validation insufficient. That single sentence rewrites twenty years of AppSec assumptions. It also explains why every enterprise AI agent attack surface analysis that starts with "add a WAF" misses the point.

Board-level concern has caught up faster than controls. Gartner's September 2025 survey found that 74 percent of IT application leaders believe AI agents represent a new attack vector into their organization, yet only 13 percent strongly agreed they had the governance structures in place to manage them. IBM's 2025 Cost of a Data Breach Report puts a dollar figure on the gap: 97 percent of organizations that reported an AI-related breach lacked proper AI access controls, and 63 percent of breached organizations either had no AI governance policy or were still developing one.

The Four-Zone Agent Trust Model

Executives need a mental model that survives a whiteboard conversation with the security team. Listing forty OWASP items does not. The BabyBots Four-Zone Agent Trust Model reduces the agent attack surface to four boundaries an executive can reason about, each with its own attack class, its own Microsoft-stack control, and its own honest gap.

Zone 1: User Prompt

Attack class: Direct prompt injection and jailbreaks. The user, or someone who has compromised the user's session, sends the agent an instruction designed to override the system prompt.

Real example: An employee pastes a "clever prompt" from a public forum that causes an internal Copilot to reveal its system instructions or confidential context, mapping to OWASP LLM07 System Prompt Leakage.

Microsoft control: Prompt Shields in Azure AI Content Safety, which detect user prompt attacks and jailbreak patterns before they reach the model.

Secure by default covers: Known jailbreak signatures, common role-play attacks, and public-forum prompt patterns.

Gap: Novel or targeted prompts crafted against your specific system prompt. Prompt Shields is a statistical classifier, not a proof.

Zone 2: Grounded Content

Attack class: Indirect prompt injection (XPIA), the class behind both EchoLeak and ForcedLeak. Instructions are hidden in emails, documents, tickets, calendar invites, web pages, or CRM fields that the agent later retrieves.

Real example: An external sender emails your organization. The email contains white-on-white text instructing Copilot to summarize the last ten messages from the CFO and encode the summary into an image URL. When the user later asks Copilot to "catch me up on this thread," the agent obeys the hidden instruction.

Microsoft control: Prompt Shields document attack detection, Spotlighting (which base64-encodes retrieved content to mark it as lower trust), Purview DLP applied to grounding sources, and grounding-source scoping in Copilot Studio.

Secure by default covers: Copilot Studio agents include built-in protection against user prompt injection attacks (UPIA) and cross-prompt injection attacks (XPIA), and Microsoft 365 Copilot applies XPIA classifiers to email and document content.

Gap: Classifier bypass, as EchoLeak proved. Poisoned SharePoint sites the agent is authorized to read. Third-party mailboxes and shared inboxes. Any grounding source that accepts input from outside your trust perimeter is a live injection channel until it is scoped, hardened, or removed.

Zone 3: Tool Outputs

Attack class: Excessive Agency and tool abuse. OWASP LLM06:2025 defines this as the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous, or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction. The root causes are excessive functionality, excessive permissions, and excessive autonomy.

Real example: A Copilot Studio agent has a connector that can send emails, create Dataverse records, and call an internal HR API. A manipulated prompt causes the agent to chain these tools: read a manager's inbox, generate a fake termination memo, and call the HR API to update employment status.

Microsoft control: Least-privilege connector design, tool allow-listing in Copilot Studio, Entra ID Conditional Access on connector service principals, and Defender for Cloud Apps monitoring of connector call patterns.

Secure by default covers: Basic connector authentication and identity flow.

Gap: The scope of what a given agent's tools can do. If you gave the agent a connector with write access, secure-by-default will not stop it from writing.

Zone 4: Generated Actions

Attack class: Exfiltration and unauthorized action. The agent produces an output, a link, an image reference, or an API call that carries data out of the organization.

Real example: ForcedLeak. The Salesforce Agentforce attack exfiltrated data to an allowlisted domain whose registration had expired and been reacquired by the attacker for the price of a coffee. The egress passed every destination check because the destination was, on paper, trusted.

Microsoft control: Purview DLP applied to Copilot outputs, trusted-URL allowlists for markdown and image rendering, Microsoft Sentinel correlation of agent egress patterns, and human-in-the-loop approval gates on high-impact actions.

Secure by default covers: Basic link redaction and Microsoft's own trusted-domain lists.

Gap: Your custom allowlists, your business-partner domains, and any destination whose trust posture can change (expired registrations, compromised third parties, shared drives with external guests).

Agent security is not a governance checkbox. It is the missing fourth pillar of the enterprise agent operating model, and it lives in security, not compliance.

The Microsoft-Stack Activation Sequence

Every CISO I talk to about defending Copilot against prompt injection asks the same question: what do I turn on, and in what order? Sequence matters because each layer assumes the one below it is working. Activating Sentinel correlation without first scoping grounding sources is like installing a smoke detector in a room full of open gasoline.

Order of Operations

1. Scope and harden grounding sources. Before turning on a single control, inventory every SharePoint site, mailbox, Dataverse table, and web source your agents can read. Remove anything they do not need. This is the highest-leverage step, and it costs nothing.
2. Enable Copilot Studio built-in UPIA and XPIA protection on every custom agent. This is a checkbox, not a project.
3. Layer Prompt Shields with Spotlighting on agents built in Azure AI Foundry or exposed through custom orchestrations. Spotlighting tags retrieved content as lower trust and materially reduces indirect injection success rates.
4. Apply Purview DLP policies to both grounding sources and Copilot outputs. Treat the agent as a data subject that must be governed like any other.
5. Enforce Entra ID Conditional Access on connector service principals and on the agent identities themselves. Recognize that AI models and agents function as identities with access privileges.
6. Route Copilot and Copilot Studio telemetry into Defender for Cloud Apps and Microsoft Sentinel. Correlate agent egress patterns with user session context to catch what individual controls miss.
7. Invoke Copilot Studio's external threat detection preview when your risk profile exceeds what built-in protections address, typically in regulated industries or for agents that touch high-sensitivity data.

The Honest Read on "Secure by Default"

Copilot Studio's UPIA and XPIA protections are real and non-trivial. Microsoft has invested heavily in classifiers, Spotlighting, and defense-in-depth patterns, and they stop the majority of commodity attacks. That is worth saying clearly, because too many security articles imply that everything Microsoft ships is either magic or marketing.

What secure-by-default does not do is cover the parts of the system you own: the scope of your grounding sources, the permissions on your connectors, the trust posture of the domains your agent can render links to, and the adversarial creativity of anyone motivated enough to bypass a statistical classifier. EchoLeak's bypass of Microsoft's own XPIA classifier is the empirical proof. Prompt Shields is a probabilistic control. Probabilistic controls fail, sometimes spectacularly, when the attacker knows they exist.

Pre-Production Agent Security Readiness

The single most useful artifact a CISO can bring to a Copilot steering committee is a one-page pre-production readiness checklist. It converts an abstract risk conversation into a defensible go/no-go decision. BabyBots uses the following as a baseline. Adapt it to your risk appetite, but do not skip items.

The BabyBots Pre-Production Checklist

• Threat model completed for the specific agent, mapped to the Four-Zone Trust Model and to MITRE ATLAS techniques.
• Grounding sources inventoried, scoped to least necessary, and hardened. External-input sources (shared mailboxes, public SharePoint, CRM Web-to-Lead fields) either removed or wrapped in additional controls.
• Tool allow-list enforced. Every connector documented with its scope, its authentication pattern, and its blast radius if abused.
• Output DLP configured. Trusted-URL allowlist reviewed for expired or compromisable domains.
• Copilot Studio UPIA and XPIA protection enabled. Prompt Shields activated where the agent is built outside Copilot Studio.
• Red-team pass documented. At minimum, one PyRIT run against the deployed agent using EchoLeak-style and ForcedLeak-style patterns.
• Human-in-the-loop escalation defined for high-impact actions. OWASP is explicit: require a human to approve high-impact actions before they are taken.
• Incident response playbook adapted. Your SOC knows what an agent compromise looks like and how to contain it.
• Named accountable owner. Not "the AI committee." A person.

Red-Teaming as a Discipline, Not a Product

Continuous adversarial testing is the control that separates organizations that learn from those that get breached publicly. Microsoft's PyRIT is an open-source framework that automates prompt-injection, jailbreak, and multi-turn attack testing against your own agents. It is free, it runs against any endpoint, and it can be scripted into your release pipeline.

The point is not the tool. The point is the discipline. Every material change to an agent, new grounding source, new connector, new system prompt, is an opportunity to reintroduce a vulnerability. Mid-market CISOs who assume red-teaming requires a dedicated AI security team should reread that sentence: PyRIT is scriptable, and one thoughtful engineer running it monthly against production agents delivers more real security than a shelf full of AI governance policies.

Three Executive Misconceptions to Reject

Misconception 1: "Microsoft handles it because it's Copilot."

Microsoft handles the model, the platform, and a bounded set of classifiers. You own the data your agents read, the tools they can call, and the domains they can render. EchoLeak was patched, but the underlying attack class, indirect prompt injection against an LLM with access to internal data and external output paths, is structural. There is no future patch that removes it.

Misconception 2: "Our existing DLP covers this."

Traditional DLP inspects egress destinations. It does not read model context, and it does not reason about tool calls as security-relevant actions. A DLP policy that blocks credit card numbers in outbound email will not notice a Copilot summarizing a confidential deal into a markdown link that renders an image to an attacker's server.

Misconception 3: "We'll add security after the pilot."

Both EchoLeak and ForcedLeak hit production deployments, not pilots. The window for adding security after go-live closed sometime in mid-2025. A pilot that reaches ten users is a production system to any attacker who finds it.

Who Owns Agent Security

The organizational-design question is the one tools do not answer, and it is where most enterprises get stuck. Agent security sits at the intersection of three functions: the CISO owns the threat model and the controls, the CIO owns the platform and the connectors, and the AI or Automation lead owns the agents themselves.

The pattern that works, in the deployments BabyBots has observed, is a single named accountable owner reporting into the CISO with a dotted line to the AI lead. This person runs the threat model, signs the pre-production readiness checklist, and holds the red-teaming cadence. They are not a committee. They are not a working group. They are the person whose name is on the incident report if it goes wrong.

The reason this reporting line matters: agent security is an adversarial discipline, not a compliance one. Governance functions optimize for policy coverage. Security functions optimize for attacker behavior. Only one of those two mindsets stops EchoLeak.

Frequently Asked Questions

What is the difference between direct and indirect prompt injection?

Direct prompt injection occurs when an attacker submits malicious instructions directly to the AI through the chat interface. Indirect prompt injection hides those instructions in content the AI will later retrieve, such as an email, a document, a web page, or a CRM record. Indirect injection is the dominant class in real-world enterprise incidents because it does not require the attacker to have direct access to the agent.

Do Copilot Studio's built-in protections stop prompt injection?

They stop many commodity attacks and materially reduce risk, but they are probabilistic classifiers, not proofs. EchoLeak bypassed Microsoft's own XPIA classifier by chaining four separate techniques. Treat built-in protections as a necessary layer, not a sufficient one, and pair them with grounding-source scoping, tool allow-listing, output DLP, and adversarial testing.

How is agent security different from AI governance?

Governance is about policy: who can build agents, what data they can use, what approvals are needed. Security is about adversaries: how would someone abuse this agent, what controls stop them, and how do we know the controls work. A governed agent can still be exploited. A secure agent has been threat-modeled and adversarially tested.

What is Excessive Agency, and why does it matter for CISOs?

Excessive Agency, defined as OWASP LLM06:2025, is the vulnerability that lets an agent take damaging actions when it malfunctions. It is caused by excessive functionality, excessive permissions, or excessive autonomy. For CISOs it matters because it converts a prompt-injection incident from a data-leak event into a data-modification or fraudulent-action event. The mitigation is least-privilege connector design and human-in-the-loop approval on high-impact actions.

How often should we red-team our production AI agents?

At minimum, before any material change to grounding sources, connectors, or system prompts. In practice, a monthly PyRIT run against production agents is a reasonable baseline for mid-market organizations. Regulated industries or agents that touch high-sensitivity data warrant more frequent testing and a formal adversarial evaluation gate in the release pipeline.

Sources

• IBM Cost of a Data Breach Report 2025, IBM and Ponemon Institute.
• Gartner Survey on Autonomous AI Agent Adoption, Gartner, September 30, 2025.
• OWASP LLM06:2025 Excessive Agency, OWASP GenAI Security Project.
• Prompt Shields in Azure AI Content Safety, Microsoft Learn.
• MITRE ATLAS, adversarial AI threat taxonomy.
• PyRIT: Open Automation Framework for Red-Teaming GenAI, Microsoft Security Blog.
• ForcedLeak: AI Agent Risks Exposed in Salesforce Agentforce, Noma Security.
• Salesforce Patches Critical ForcedLeak Bug, The Hacker News, September 25, 2025.

The Strategic Implication

Agent security is the earliest test of whether your organization can absorb a fundamentally new class of risk without waiting for a public incident to force the conversation. The organizations that win the next five years will be the ones that treated the first Copilot rollout as a security architecture project, not a productivity project. The ones that lose will be the ones whose first serious agent incident arrives before their first serious threat model.

The Microsoft stack gives you nearly every control you need. BabyBots has watched enterprises succeed and fail at activating them, and the pattern is consistent: the technology is not the constraint. Ownership, sequencing, and adversarial discipline are. Fix those, and the fourth pillar of your agent operating model is in place before your competitors realize they are missing it.