AI & Infrastructure
(RAG) Retrieval Augmented Generation

(RAG) Retrieval Augmented Generation

Retrieval-augmented generation (RAG) is an AI architecture that fetches relevant documents at query time and adds them to a language model’s prompt. The model then answers from those sources and can cite them.
June 27, 2026
Cihan Geyik
Table of Content

A language model on its own can’t cite anything. It keeps no record of where its knowledge came from. Retrieval-augmented generation (RAG) changes that by fetching documents first and writing second. Nearly every AI search product works this way, which makes RAG the step that turns a ranking page into a cited source.

No retrieval, no citation. In a RAG system the model writes from the passages it was handed. A page that isn’t retrieved can’t be quoted or linked, however good it is.

What is retrieval-augmented generation?

The term comes from a 2020 paper by Patrick Lewis and co-authors, presented at NeurIPS that year. They combined two kinds of memory. Parametric memory is what a pretrained model stores in its weights. Non-parametric memory was a dense vector index of Wikipedia, searched by a neural retriever.

The authors named two problems that outside memory could solve. One was showing where an answer came from. The other was updating knowledge without retraining. Answer engines adopted RAG for the same two reasons.

How does RAG work?

Split documents into chunks → Index them → Retrieve chunks for the query → Rerank → Add the best ones to the prompt → Generate an answer with citations

Systems differ in the details. Some retrieve with keyword search, some with vector search, and many combine the two. Web answer engines also rewrite the user’s prompt into several searches before retrieving. The step-by-step sequence is covered under retrieval pipeline.

Newer systems retrieve in loops. An agentic RAG system checks whether it has enough evidence and searches again when it doesn’t. Deep research modes in AI assistants work this way, running many searches for one report. See multi-step retrieval for how those rounds work.

RAG vs grounding vs fine-tuning

TermWhat it isCan it cite sources?
RAGAn architecture: retrieve documents, then generateYes
GroundingThe aim: answers tied to verifiable sourcesYes, when it runs through retrieval
Fine-tuningExtra training that changes the model’s weightsNo
Long contextWhole documents placed in a large windowOnly if the system tracks where text came from

Grounding is usually achieved through RAG. Fine-tuning is a different tool, better at teaching format and tone than facts.

Why does chunking decide what gets cited?

RAG systems split documents into chunks before indexing, and a chunk loses the text around it. Anthropic’s contextual retrieval write-up gives an example. A chunk says a company’s revenue grew 3% over the previous quarter, but names neither the company nor the period. A retriever can’t match it to a question about that company.

Anthropic’s fix was to add a short context line to each chunk before indexing. Paired with contextual keyword matching, that cut top-20 retrieval failures by 49%. Adding reranking took the cut to 67%.

Publishers can apply the same idea in their writing. Every section should name its subject and state its main fact, so it still makes sense after it’s cut loose from the page.

Why does RAG matter for AI visibility?

AI search engines are RAG systems at web scale. ChatGPT search, Perplexity, AI Overviews and Copilot search first, then write from what they found. Each page has to clear three gates.

  1. Be crawlable and indexed by the engine’s search source.
  2. Be retrieved for the searches the engine runs for the prompt.
  3. Offer a passage good enough to use and cite.

Our own site shows the scale involved. ansvisor.com was a cited source in 9.6% of the 103,427 AI answers in our dataset. Source: Ansvisor tracking data, September 15 to October 14, 2026. Each of those citations began with a retriever selecting one of our pages. Our guide on how to get cited in ChatGPT and AI search covers the page-level work.

Common RAG failure modes

  • The retriever misses the best source because the page uses different words than the query.
  • A chunk is retrieved without the detail that makes it true, such as the year or the plan tier.
  • Two conflicting sources get blended into one wrong statement.
  • The citation points to a page, but the claim came from a different passage or site.

How Ansvisor helps

Ansvisor’s Citations Monitoring lists the URLs AI answers cite, filtered by prompt, topic, platform, region and language. Its gap view shows prompts where competitor pages get cited and yours don’t. Query Fan-Out adds the searches ChatGPT, Perplexity and Microsoft Copilot ran, so you can see what the retrieval step looked for. Start a 14-day free trial to see the sources behind your own prompts.

RAG, Retrieval-Augmented Generation, Retrieval Augmented Generation, Retrieval-Augmented LLM, Grounded Generation

Let AI Search Work for Your Growth

Each prompt you win brings more mentions, more citations and more buyers. Ansvisor keeps finding the next one.

AI Answers Mentions Citations Visibility Score AI Share of Voice AI Traffic Revenue
Win Back Search VisibilityTry Ansvisor 14-day Free →✓ Zero risk. Cancel anytime.

FAQ

Frequently asked questions.

What is retrieval-augmented generation (RAG)?

RAG is a way of building AI systems in two steps. A retriever finds documents relevant to a question, and a language model writes the answer from them. It lets the model use current or private information it never saw in training, and it makes citations possible.

How does RAG work?

Documents are split into chunks and indexed, often as embeddings. When a question arrives, the system retrieves the most relevant chunks, adds them to the prompt and asks the model to answer from them. Many systems also rerank results before generation and attach citations after it.

What is the difference between RAG and fine-tuning?

RAG adds outside documents to the prompt at answer time and leaves the model unchanged. Fine-tuning changes the model’s weights with extra training. RAG suits facts that change and answers that need sources, while fine-tuning suits style, format and narrow behaviors.

Is ChatGPT search a RAG system?

It follows the same pattern. ChatGPT search, Perplexity, Google AI Overviews and Microsoft Copilot run web searches, read the results and write answers that link to their sources. Each uses its own index, ranking and citation rules, so the pages they pick differ.

Why does RAG matter for SEO and AI visibility?

In a RAG-based answer engine, a page can be cited only if the retriever finds it and selects one of its passages. Crawl access, indexing and self-contained passages become conditions for appearing in AI answers, next to the authority signals SEO already works on.

Help us grow the AI Visibility Glossary

New terms are added regularly.

Help us improve the page or suggest a new term →
About the Author
Cihan Geyik

Cihan Geyik

Co-founder at Ansvisor

Cihan Geyik is the co-founder of Ansvisor, an open-source AI Visibility platform for AI Search. With more than 15 years of experience in digital marketing and growth, he writes about AI visibility, AI search, AEO, GEO, citations, and answer engines. He focuses on helping brands understand and improve their presence across ChatGPT, Gemini, Google AI Overviews, Bing AI Mode and other AI answer engines.

Google Preferred Sources
Get more Ansvisor insights in Google

Add Ansvisor as a preferred source and Google will show you more of our AI search, AEO and GEO guides.