Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, driven by escalating costs, unclear business value, and inadequate risk controls. In our work with enterprise Copilot Studio programs, we see the same failure pattern behind almost every stalled pilot: the agent hallucinates, users lose trust, and the program quietly gets defunded. Leaders assume the model is at fault. It almost never is. The failure is AI agent knowledge grounding: which sources are attached, how they are structured, and who is operating them.

Grounding is not a technical checkbox. It is a knowledge-operations discipline. And in the Microsoft stack, choosing between SharePoint, Dataverse, and website grounding is the single most consequential architectural decision an agent builder makes.

TL;DR

Grounding anchors an AI agent's answers in verified enterprise data instead of the model's parametric memory. In Copilot Studio, the source you attach determines whether the agent tells the truth, hallucinates confidently, or leaks data across permission boundaries. SharePoint, Dataverse, and website grounding solve different problems, and using them incorrectly is why "grounded" agents still hallucinate.

Key Takeaways

• Hallucination in enterprise agents is usually a knowledge-operations problem, not a model problem.
• SharePoint, Dataverse, and website grounding differ on freshness, permissions fidelity, structure fit, and governance overhead. They are not interchangeable.
• Copilot Studio's RAG pipeline has four steps, and grounding failures happen at every one of them.
• Azure AI Search does not enforce user permissions by default. Treat it as a governance trap, not a shortcut.
• RAG is the wrong pattern for complex reasoning, cross-document synthesis, and policy compliance evaluation. Use actions and agentic workflows instead.

Why Grounded Agents Still Hallucinate

Most enterprises assume that attaching a knowledge source is what stops hallucination. It is not. Grounding does not eliminate hallucination. It moves the failure surface from the model to your knowledge estate.

Once that surface moves, the failure modes are operational, not algorithmic. Documents are stale. Titles are ambiguous. Files exceed size limits and get silently skipped by the indexer. Permissions are misconfigured, so security trimming quietly hides the right answer. Chunks are low quality because the source PDF was scanned, not text-based. The model does exactly what it should. The knowledge estate does not.

This is why the same LLM produces a trustworthy agent in one department and a liability in another. The model is a constant. The knowledge estate is not.

The Copilot Studio RAG Pipeline in Under 250 Words

To make grounding decisions well, builders need to understand what Copilot Studio actually does when a user asks a question. Microsoft documents a four-step RAG pipeline:

1. Query rewriting. The user's message is reformulated into a search-optimized query, expanded with conversation context. A vague question here means poor retrieval downstream.
2. Content retrieval. The rewritten query runs against every attached knowledge source in parallel. Copilot Studio pulls the top three results from each source, balancing relevance and latency. Each source has its own retrieval mechanics: SharePoint uses Microsoft Search with Entra-delegated permissions, Dataverse translates natural language into structured queries, and Bing indexes public content with limited depth.
3. Summarization and response generation. The model composes an answer from the retrieved chunks and produces citations.
4. Safety and governance validation. Responsible AI filters and, if enabled, the "ungrounded responses off" setting withhold answers that lack a citation.

The point is not the mechanics. The point is that hallucination can originate at any step. A bad rewrite. A missed retrieval. A model that generates a correct answer but forgets to cite. Understanding this is what separates practitioners who fix agents from those who blame them.

SharePoint vs Dataverse Grounding vs Website Sources

The three sources most Copilot Studio agents rely on behave very differently. Below is how they compare on the dimensions that actually determine agent quality in production.

Grounding Source Comparison

SharePoint and OneDrive

• Best for: Unstructured policy, procedure, FAQ, and reference content.
• Freshness: Sync every four to six hours after ingestion completes.
• Permissions fidelity: High. Entra ID delegated identity, with security trimming.
• Hard constraints: 7 MB file cap without an M365 Copilot license in tenant; up to 200 MB with Enhanced Search Results and tenant graph grounding.
• Failure mode: Sprawl. Attaching an entire site inherits duplicates, outdated policies, and ambiguous filenames.
• Governance overhead: High. Requires ongoing content curation.

Dataverse Tables

• Best for: Structured record lookup and natural-language analytical queries over business data.
• Freshness: Effectively real-time.
• Permissions fidelity: High. Entra-delegated with row-level security respected.
• Hard constraints: Up to 15 tables per knowledge source, and quality collapses without synonyms and glossary terms.
• Failure mode: Under-modeling. The agent cannot translate "open opportunities" or "AP invoices" without the glossary that defines them.
• Governance overhead: Medium. Data owners must maintain synonyms as the business vocabulary evolves.

Public Website and Bing Custom Search

• Best for: External-facing agents grounded in already-public content.
• Freshness: Dependent on Bing's index cycle.
• Permissions fidelity: Not applicable, all content is public.
• Hard constraints: Two-subpage indexing depth; Bing Custom Search supports up to 400 URLs with Azure costs absorbed by Microsoft.
• Failure mode: Content behind authentication is invisible. Marketing sites often hide the real answer behind a gated PDF.
• Governance overhead: Low, but only because the content is already governed by web publishing.

Azure AI Search

• Best for: Complex vector search across large, heterogeneous corpora.
• Permissions fidelity: None by default. The connection is not delegated, so every user sees whatever is indexed.
• Failure mode: Silent data exposure. Treat this as a governance decision, not a technical shortcut.

The BabyBots Grounding Fit Matrix

After running these decisions across dozens of Copilot Studio deployments, we now use a simple diagnostic with every client. The Grounding Fit Matrix maps a candidate use case to the correct source by asking four questions in order. Skipping any one of them is where most agent programs go wrong.

The Four Questions

1. What shape is the data? Unstructured documents point to SharePoint or Graph connectors. Structured records point to Dataverse or real-time connectors. Mixed corpora point to Azure AI Search, with governance guardrails.
2. Who is allowed to see it? If the answer varies by user, you need a source with delegated identity. That eliminates Azure AI Search and public website grounding.
3. How fresh must the answer be? Sub-hour freshness eliminates SharePoint and points to Dataverse or a real-time connector.
4. Who will own the source in month twelve? If nobody can answer this, the source will decay. Reassign or defer.

The order matters. Data shape is a design question. Permissions is a risk question. Freshness is an SLA question. Ownership is the operating question, and it is the one most teams skip. An agent with a great architecture and no owner will hallucinate within a quarter.

Fixing the model can't fix a grounding gap. Fixing the knowledge estate can.

When Not to Use RAG at All

Retrieval-augmented generation is a fact-lookup pattern. Microsoft is explicit that RAG is not intended for full document comparison, policy compliance evaluation, or complex reasoning over long unstructured documents. Yet these are precisely the use cases enterprises attempt first, because they are the highest-value problems in the room.

When the task requires reasoning across sources, chaining decisions, or executing an action, the right pattern is not a bigger knowledge source. It is an agentic workflow: tools, actions, and orchestrated steps that use RAG for facts and code for logic. If your pilot is asking an agent to "compare our contract against the vendor's redlines and flag risks," you are not solving a grounding problem. You are solving a workflow problem, and no amount of SharePoint tuning will get you there.

Knowing when to stop grounding and start orchestrating is one of the highest-leverage skills a Copilot Studio program can develop.

Grounding Is Knowledge Operations, Not IT Admin

The organizations that convert agent pilots to production treat grounding as a continuous operating discipline. The ones that contribute to Gartner's 40% cancellation statistic treat it as a setup task. The difference shows up in five practices.

• Content curation. Someone owns the decision about what belongs in the agent's knowledge source and what does not. Sprawl is a choice.
• Freshness SLAs. Each source has a defined maximum staleness, monitored and reported.
• Source retirement. Retired policies, deprecated products, and obsolete procedures are removed from grounding scope on a schedule, not on complaint.
• Grounding QA. A recurring evaluation set of representative questions is run against every production agent, with citation quality scored.
• Ownership. A named business owner is accountable for the truthfulness of each agent, distinct from the IT owner of the platform.

None of this is glamorous. All of it is the difference between a pilot and a program.

The Grounding Operations Maturity Curve

To help executives benchmark their program honestly, we use a five-level maturity curve. Most enterprises we assess are at Level 1 or Level 2 and are surprised to learn it.

Levels 1 Through 5

• Level 1: Attached. A SharePoint site or Dataverse table has been connected. Nobody owns it. Nobody measures answer quality. Hallucination is blamed on the model.
• Level 2: Curated. Sources have been narrowed to relevant content. Oversized files have been split. Basic synonyms exist for Dataverse.
• Level 3: Governed. Each source has a named owner, a freshness SLA, and a retirement process. Permissions are validated. Ungrounded responses are off.
• Level 4: Measured. A recurring grounding QA process scores citation quality and answer accuracy. Failures feed a content backlog, not a model complaint.
• Level 5: Operated. Grounding is a formal knowledge-operations function, with cross-agent governance, source lifecycle management, and executive reporting on trust metrics.

Progression is not automatic and it is not linear. Most organizations plateau at Level 2 because the leap to Level 3 requires assigning ownership across IT and the business, which is an organizational decision, not a technical one.

Frequently Asked Questions

Why is our "grounded" Copilot Studio agent still hallucinating?

Almost always because the right content was not retrieved. Common causes include files that exceed the 7 MB SharePoint limit and were silently skipped, missing Dataverse synonyms, ambiguous document titles, stale content that outranks the correct version, or the model returning a correct answer without a citation and being suppressed by the ungrounded-responses guardrail. Diagnose retrieval before you blame the model.

Should we use SharePoint or Dataverse for our agent's knowledge?

Use SharePoint for unstructured content people read: policies, procedures, guides, FAQs. Use Dataverse for structured business records the agent needs to query: accounts, cases, transactions. The two are complementary. Most production agents ground on both, with each source scoped tightly to what it does best.

Is Azure AI Search a safe grounding choice?

Only with explicit governance. Azure AI Search does not enforce user-level permissions by default, so anything indexed becomes visible to every agent user. It is powerful for vector search across heterogeneous content, but treat the permissions model as a design decision, not an afterthought.

Who should own grounding, IT or the business?

Both, in defined roles. IT owns the platform, connectors, and security model. The business owns the content, the vocabulary, and the truthfulness of the answers. A named business owner per agent is the single strongest predictor of grounding quality we have observed.

When should we not use RAG at all?

When the task requires reasoning across sources, comparing documents, evaluating policy compliance, or executing an action. Those are workflow problems. Use actions, tools, and agentic workflows, with RAG reserved for the fact-lookup steps inside them.

Sources

• Enhance AI responses with Retrieval Augmented Generation, Microsoft Learn (Copilot Studio).
• Knowledge sources summary, Microsoft Learn (Copilot Studio).
• Quotas and limits, Microsoft Learn (Copilot Studio).
• Add Dataverse tables as a knowledge source, Microsoft Learn (Copilot Studio).
• Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, Gartner, June 2025.

The Executive Takeaway

Enterprise agent programs will not be won or lost on model selection. They will be won or lost on the discipline with which organizations operate their grounding estate. The companies that treat SharePoint, Dataverse, and website grounding as distinct instruments, assign named owners, and run grounding QA as a recurring practice will convert pilots to production. The companies that connect a site and hope will contribute to Gartner's cancellation statistic.

At BabyBots, we help organizations move from Level 1 to Level 3 grounding maturity as part of broader Copilot Studio and process transformation programs. The technology is ready. The knowledge estate rarely is. Closing that gap is the work.