What is RAG · Retrieval-Augmented Generation in Generative AI | Generative AI & LLMs

What is RAG

Give the Model the Facts

Retrieval-Augmented Generation (RAG) means: before asking the model, retrieve the most relevant pieces of your own documents and put them in the prompt. The model then answers from those facts. This is how support bots answer from your policies, and how tools like Perplexity answer from web pages.

OFFLINE  policy docs ──► chunk ──► embed ──► vector store

ONLINE   question ──► embed ──► top-k chunks ──► prompt = instructions + chunks + question
                                                         │
                                                  LLM ──► answer with citations

Why RAG Instead of Retraining

Need	RAG
Private data (policies, orders, manuals)	Yes: send only what is needed, per request
Data changes daily	Update the index; no training
Show sources	You know exactly which chunks were used
Access control	Filter chunks by the user's permissions before the model sees them
Reduce hallucination	The model is told to answer only from the context

Tip: With 1M-token context windows you can sometimes paste a whole small document instead. RAG is still cheaper and faster when there are many documents or many requests.