MCP vs RAG: retrieval vs tools, and when an agent needs both

RAG (retrieval-augmented generation) finds relevant text and puts it in front of the model before it answers. MCP (the Model Context Protocol) is a standard way for an AI app to connect to outside systems, so the model can call tools and read data while it works. RAG is about what the model knows. MCP is about what it can reach and what it can do. They aren't competitors, and RAG is increasingly exposed over MCP, as a search tool the model decides to call.

That last point is why many "MCP vs RAG" comparisons feel slightly off. They line the two up as rival architectures: RAG for static documents, MCP for live APIs. In practice they sit at different layers. RAG is a pattern, MCP is a protocol, and the useful question is which parts of your agent's job are lookups and which are actions. A phone call is a good test case. You can retrieve the dentist's number. You can't retrieve a rescheduled appointment.

MCP vs RAG at a glance

What RAG is

The term comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, UCL and NYU, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks". They paired a generator model with a retriever over a dense vector index of Wikipedia, so the model could draw on passages it hadn't memorized. The name stuck long after the specific architecture stopped mattering.

Today "RAG" usually means a pipeline like the one Anthropic describes in its Contextual Retrieval post:

1. Split the documents into chunks, usually no more than a few hundred tokens each.
2. Turn each chunk into an embedding and store it in a vector database. Many systems also build a keyword index (BM25) so exact strings like an error code still match.
3. At question time, find the chunks closest to the question and add the best ones to the prompt.
4. The model answers using those chunks, ideally citing them.

Two details from that post are worth keeping in mind. First, chunking strips context: a chunk saying "revenue grew 3%" may not say which company. Anthropic reported that prepending context to each chunk before embedding it and building the BM25 index cut top-20 retrieval failures by 49%, and by 67% when a reranker was added. Second, RAG is not always needed. If the knowledge base is under about 200,000 tokens, roughly 500 pages, Anthropic's 2024 post suggests putting the whole thing in the prompt and using prompt caching instead.

What MCP is

MCP is an open protocol Anthropic released in November 2024 and donated to the Agentic AI Foundation in December 2025. An AI app (the host) runs MCP clients, each connected to a server. A server offers three kinds of thing. Tools are functions with an input schema, and the current spec calls them "model-controlled": the model discovers them and decides when to call them. Resources are data identified by a URI, and the spec calls them "application-driven": the host app decides how to bring them into context. Prompts are templates the user picks. Our tools, prompts and resources guide covers who controls each one.

Nothing in MCP says how a server finds information. A server can wrap a REST API, run SQL, read files, or query a vector index. That last option is where the two ideas meet.

Why "MCP or RAG" is usually the wrong question

RAG answers "how does relevant text get into the prompt?" MCP answers "how does an AI app talk to outside systems in a standard way?" Those questions don't compete. You can run RAG with no MCP at all, which is how RAG worked before MCP existed. You can run MCP with no retrieval at all, which is how a calendar or phone-call server works. And you can run retrieval through MCP.

Qdrant's official MCP server is a clean example. It exposes two tools, qdrant-store and qdrant-find. The second one takes a query string (plus a collection name, unless a default is configured) and returns matching entries from a vector collection, embedded by default with sentence-transformers/all-MiniLM-L6-v2. To the model it's just a tool. Here is the call, with the required _meta fields left out for brevity:

{
  "jsonrpc": "2.0", "id": 3, "method": "tools/call",
  "params": {
    "name": "qdrant-find",
    "arguments": { "query": "dentist cancellation policy" }
  }
}

That is RAG. The retrieval step and the vector index are all there. What changed is who triggers it (the model, mid-task, instead of your pipeline, up front) and how it's wired (one MCP server any compatible client can use, instead of retrieval code baked into one app). Setting QDRANT_READ_ONLY removes the store tool, which leaves a pure retrieval server.

So a better framing is two separate decisions. Is this piece of information something to look up, and if so, in an index or in the live system? And does the agent need to do something, with a side effect in the world?

The difference that holds up: lookups vs actions

Common advice says RAG handles unstructured, slow-changing data and MCP handles structured, live data. That's a fair description of habits, but it isn't built into either one. You can index a database export for RAG, and an MCP server can search PDFs. The distinction that does hold up is simpler.

Retrieval can only return what was written down before. An index holds documents someone already wrote. A live query returns a record that already exists. Both are lookups, and a lookup never changes the world.

An action creates a fact that didn't exist before you acted. A booking, a sent message, a refund, a phone call where the clinic agrees to move you to Thursday. No index contains the answer to "will they move my appointment?" until someone asks them. For that the agent needs a tool with a side effect, and MCP is the common way to give it one.

That's also why tools carry risks that retrieval doesn't. A wrong retrieval gives you a wrong answer. A wrong tool call can send an email or book the wrong time. The MCP spec says there should always be a human able to deny a tool call, and our MCP security checklist goes through the controls a server should enforce itself.

Agentic RAG: retrieval as a tool

"Agentic RAG" is the name for the setup in the Qdrant example. Instead of a fixed retrieve-then-generate step, an agent decides whether to search, what to search for, and whether the results are good enough or it should search again. A January 2025 survey, Singh et al., describes it as putting agents inside the RAG pipeline, using planning, reflection, tool use and multi-agent patterns to manage retrieval.

Once retrieval is a tool, MCP is a natural way to expose it. The same search server works in Claude, Cursor, VS Code or your own agent, and it sits next to the tools that act. The trade-offs move around rather than vanish:

• Better: the model can rephrase a failed query, search two sources, or skip retrieval when the question doesn't need it.
• Worse: every search is a model turn, so latency and token use go up, and the model can also decide not to search when it should have.
• Same: the index still has to be built, chunked well and kept fresh. MCP doesn't fix bad chunking.

For a question-answering bot over a fixed manual, classic retrieve-then-generate is often still the better choice: cheaper, faster and easier to evaluate. Agentic retrieval earns its cost when questions are open-ended or when retrieval is one step in a longer task.

Which one you need, by job

If you're choosing between MCP and calling a REST API directly from your own code, that's a different question, covered in MCP vs API. And if the other side is a whole agent rather than a tool, see A2A vs MCP.

A worked example: an agent that needs both

Say you ask your assistant: "Move my dental cleaning to the week after next, any afternoon, and make sure it's still covered." Here's how the work splits.

1. Retrieval. The agent searches your own notes and documents for the dentist's details, your current appointment, and the insurance plan's rule on cleanings. Those are lookups over text you wrote or saved, which is what RAG is good at. The search can be a local index or an MCP search server like the Qdrant one.
2. Judgment. The model reads what came back and works out what to ask for: afternoons the week of November 9, and a check that the plan's cleaning frequency allows this date.
3. Action. No index knows whether the clinic has a Tuesday 3pm slot. Someone has to ask. With DialMCP connected, the agent calls place_call with an objective built from what it retrieved:

{
  "jsonrpc": "2.0", "id": 8, "method": "tools/call",
  "params": {
    "name": "place_call",
    "arguments": {
      "to": "+14155550123",
      "callee_name": "Example Dental",
      "objective": "Move Dana's cleaning from Thu Nov 5 at 10:00 to any afternoon the week of Nov 9 (Nov 9-13). Accept 13:00-17:00. Confirm the new date and time back.",
      "context": "Date of birth if asked: 1990-04-12. Insurance: same plan as on file."
    }
  }
}

4. Follow-up. The call runs in the background, typically two to five minutes. The agent polls get_call until it ends and reads the resolution: an outcome, a short summary, commitments (the new slot), follow-ups and key facts.
5. Write-back. The new appointment is now a fact that exists, so it can go back into your notes, where the next retrieval will find it.

Steps 1 and 5 are retrieval and storage. Step 3 is the part no retrieval system can do. Step 2 is the model. Cutting any one of them makes the errand worse. Without retrieval the call goes out missing details. Without the tool, the agent can only tell you what to say when you call.

One honest caveat: DialMCP's resolution is written by a model reading the transcript, and speech recognition can mishear names and numbers. For a confirmation code, check the transcript or recording. The tool reference covers that in full.

What each one costs you to run

RAG's costs are mostly up front and ongoing: picking a chunk size, an embedding model and an index, re-indexing when documents change, and keeping access control in sync so one user's search can't surface another user's files. Quality is hard to see. A missed retrieval looks like a confident wrong answer, so you need test questions with known answers.

MCP's costs show up at call time: tool definitions and tool results take up context, remote servers typically need OAuth, and every tool with a side effect needs a clear approval story. Tool design matters as much as chunking does for RAG. A vague schema produces vague calls; see MCP tools explained.

Shared risk: retrieved passages and tool results are both text from outside the model. Either can carry a prompt injection. A poisoned document in your index and a poisoned tool result are the same problem in different places, and neither RAG nor MCP solves it by itself.

Does one perform better?

There's little head-to-head evidence, partly because they do different jobs. One useful data point is a November 2025 technical report (not peer reviewed), "MCP vs RAG vs NLWeb vs HTML". Its authors built four simulated online shops and gave agents the same tasks, from product search to checkout, through four interfaces. RAG, MCP and NLWeb agents all beat plain HTML browsing: average F1 rose from 0.67 to between 0.75 and 0.77, and token use fell from about 241,000 per task to between 47,000 and 140,000. The best single setup was RAG with GPT 5 (F1 0.87).

Read that as "a structured interface beats scraping pages," not as a general verdict. In that study, RAG meant search over pre-crawled shop pages and MCP meant the shops' web APIs. These were shop tasks with a known catalog, on simulated sites, and results also varied by model.

Where DialMCP fits

DialMCP is on the action side of the line. It's a remote MCP server with four tools for placing and following calls, and no search index. Your agent brings the knowledge, from RAG, memory, or just your message, and DialMCP makes the phone call from your own verified number. The voice agent says it's an AI assistant and that the call is recorded, can work through phone menus and wait on hold, and is told to state only facts given in the objective or context, plus your first name and callback number. That last rule is a reason to get retrieval right before dialing: the call can only be as specific as what the agent found.

For "what happened on that call yesterday?", list_calls returns your recent calls, newest first, with their outcome. That's a lookup, so if you want months of call summaries to be searchable, index the resolution summaries into whatever retrieval system your agent already uses.

FAQ

What is the difference between MCP and RAG?

RAG (retrieval-augmented generation) is a pattern: find relevant passages in an index and add them to the prompt so the model answers from them. MCP (the Model Context Protocol) is a standard for connecting AI apps to outside systems, so the model can call tools and read data while it works. RAG improves what the model knows; MCP extends what it can reach and do, including actions with side effects.

Does MCP replace RAG?

No. They work at different layers, and many systems use both. A RAG search can even be exposed as an MCP tool, so any MCP client can call it. What MCP adds is a standard way to reach live systems and take actions, which retrieval alone can't do.

Can you do RAG over MCP?

Yes. An MCP server can expose a search tool backed by a vector index. Qdrant's official MCP server, for example, offers a qdrant-find tool that runs a semantic search and returns matching entries. The model decides when to call it, which is usually called agentic RAG.

What is agentic RAG?

Agentic RAG puts an agent in charge of retrieval instead of running a fixed retrieve-then-generate step. The agent decides whether to search, what to search for, and whether to search again. It can handle open-ended questions better, at the cost of more model turns, latency and tokens.

Do I need a vector database to use MCP?

No. MCP doesn't care how a server finds information. Many MCP servers call APIs or databases directly and never touch embeddings. You only need a vector index if you want semantic search over a large body of text, and then it can sit behind an MCP tool or in your own retrieval pipeline.