Retrieval-augmented generation (RAG) is a technique where an AI model first searches a defined set of documents for relevant passages, then writes its answer using those passages as evidence. For marketers it means chatbots, assistants and content tools that answer from your own product sheets, policies and research instead of from the model’s general memory. It reduces invented answers, though it does not remove them.
How does RAG work, step by step?
The idea was described in a 2020 research paper from Facebook AI Research and has since become the standard pattern for business AI assistants. It runs in two phases.
Preparation happens once and is refreshed as documents change. Documents are split into chunks, each chunk is converted into an embedding, a list of numbers representing its meaning, and stored in a vector database or search index.
Answering happens every time someone asks a question:
- The question is converted into an embedding.
- The system retrieves the chunks with the closest meaning, often combined with keyword search.
- Those chunks are inserted into the prompt with instructions to answer only from them.
- The model writes an answer, ideally with references to the chunks used.
Why not just use a chatbot as it is?
General models know a lot, but not your price list, return policy or latest product release. Asked about them, they may produce plausible but wrong details. Fine-tuning a model on your documents is expensive and goes stale quickly.
RAG keeps knowledge outside the model. Update a document and the next answer reflects it. You can also show sources, which lets users and reviewers check the claim.
Where does RAG fit in marketing work?
The best uses share two features: a defined body of trusted content and repeated questions about it.
- Website assistants answering product, pricing and policy questions from official pages.
- Sales enablement: reps asking about case studies, features or competitor comparisons from approved documents.
- Content production: drafting articles grounded in the company’s own research, product documentation and brand guide.
- Internal search across campaign reports, briefs and meeting notes.
- Customer support deflection, where answers must match the help centre.
It is a weaker fit for open creative work, such as campaign ideas, where grounding in existing documents can limit originality.
Which tools can marketers use without building from scratch?
You do not need engineers for every RAG project. Many tools include retrieval out of the box:
- Custom GPTs and ChatGPT projects with uploaded files.
- Claude Projects with a knowledge base.
- Google NotebookLM for answering from uploaded sources with citations.
- Microsoft Copilot Studio agents grounded in SharePoint or websites.
- Help desk platforms such as Intercom and Zendesk with AI agents that use the help centre.
For larger or customer-facing systems, teams build with frameworks such as LangChain or LlamaIndex and vector stores like Pinecone, Weaviate or pgvector, or use managed services from cloud providers.
What makes a RAG system good or bad?
Most quality problems come from retrieval, not the model. If the right passage is not retrieved, the model cannot use it.
Content quality
Outdated, duplicated or contradictory documents produce contradictory answers. Clean the source set before indexing. One current pricing page beats five old PDFs with different prices.
Chunking and metadata
Chunks that are too small lose context; too large and they dilute relevance. Adding metadata such as product, region and date lets the system filter before searching.
Instructions and refusal
Tell the model to say it does not know when the documents do not cover the question. An assistant that admits a gap is safer than one that guesses.
How do you evaluate a RAG assistant?
Build a test set of 50 to 100 real questions with the correct answer and the document that contains it. Then measure:
- Retrieval hit rate: did the right document appear in the retrieved chunks?
- Answer accuracy against the reference.
- Faithfulness: does the answer stay within what the sources say?
- Refusal accuracy: does it decline questions the documents do not answer?
Run the test whenever documents, chunking or prompts change. Frameworks like Ragas can automate parts of this, but human review of a sample remains essential.
How Alien Road applies RAG in AI optimisation work
A typical engagement begins with a content audit: which documents are authoritative, which are outdated, and who owns each one. That step often matters more than the technology choice, because a RAG assistant inherits whatever is in its sources.
Setup picks the lightest tool that fits. An internal sales assistant may be a Claude Project or NotebookLM notebook; a customer-facing website assistant needs a proper build with retrieval logging, refusal rules and a handover to a human. The test question set is written with the client’s sales or support team.
Measurement tracks retrieval hit rate, answer accuracy and refusal accuracy on the test set, plus live metrics such as questions answered without escalation. Monthly reports list the most common unanswered questions, which usually point to missing or unclear content on the client’s own site.
How does RAG connect to search visibility?
AI search engines such as Perplexity, Google’s AI Overviews and ChatGPT search use retrieval in a similar way: they fetch pages, then write answers from them. Content that is clear, well structured and easy to quote tends to be retrieved and cited. Building your own RAG system is a practical way to discover how easy your content is for machines to use.
What does a RAG project typically cost?
Costs fall into three buckets. Content work comes first: cleaning, updating and deduplicating documents, which is staff time. Tooling comes second: a no-code option like NotebookLM or a Claude Project costs little beyond existing subscriptions, while a custom customer-facing build adds development, hosting and a vector database.
Running costs come last, mainly model calls per question and the extra tokens from retrieved passages inserted into each prompt. For most internal assistants these stay small. For a public website assistant with heavy traffic, estimate monthly questions and multiply by the average tokens per answer before choosing a model.
Where to start this week
Collect the 20 documents your sales or support team uses most and remove outdated versions. Upload them to NotebookLM or a Claude Project and ask the 30 questions customers ask most. Note every answer that is wrong or missing. Those gaps show if you need a RAG tool, better content, or both.
Keep reading: AI Optimization · AI Optimization