Retrieval-augmented generation (RAG) is a method where a language model first searches your own documents and then writes an answer based on what it found. It beats traditional site search, static FAQs and rule-based chatbots when questions are varied and your content is large. It loses when questions are few and predictable, when answers must be exact, or when your source content is messy and out of date.
How does RAG work in plain terms?
A RAG system has three parts. First, your documents are split into chunks and turned into embeddings, which are numerical representations of meaning, stored in a vector database or search index. Second, when a user asks a question, the system finds the chunks most similar to it. Third, those chunks are passed to a language model with instructions to answer using only that material.
The appeal is that the model can answer from current, private information it was never trained on, such as your product manuals, policy documents or help centre. The answer can also cite which document it came from, which makes checking easier.
What are the traditional tools it replaces?
Before comparing, it helps to name what most companies already use.
- Keyword site search, such as WordPress search, Algolia or Elasticsearch, which returns a list of matching pages.
- Static FAQ pages and help centres built in Zendesk, Intercom or a CMS.
- Rule-based or decision-tree chatbots that follow scripted flows with buttons.
- Human support agents working from an internal knowledge base.
Each of these is cheaper to run and easier to predict than RAG. The question is whether your users get what they need from them.
When should you switch to RAG?
RAG pays off when the gap between what users ask and how your content is organised is large. A few signs:
Users ask in their own words
If your search logs show many queries that return nothing or the wrong page, keyword search is failing on vocabulary. RAG matches meaning, so “can I return a sofa after two weeks” finds the returns policy even if it never uses the word sofa.
Answers are spread across documents
When a correct answer needs a sentence from the warranty, another from the shipping page and a figure from a product sheet, a search result list makes the user do the assembling. RAG can combine them.
Your content is large and changes often
Rule-based bots need someone to update every flow by hand. RAG reads the latest version of your documents once they are re-indexed, so maintenance shifts from writing scripts to keeping source content correct.
When should you not switch?
There are clear cases where traditional tools are the better choice.
If your top 20 questions cover most of your traffic, a well-written FAQ page and a scripted bot will answer them faster, cheaper and with no risk of an invented answer. Look at your support tickets before building anything.
If answers must be exact and legally binding, such as tariff rates, medication dosages or contract terms, a generated paraphrase is a liability. Link to the authoritative page instead, or show the source text verbatim.
If your documents contradict each other, RAG will reflect the contradiction, sometimes choosing the wrong version. Cleaning the content is a prerequisite, not an optional extra.
And if nobody owns the content, the system will decay. Outdated documents in the index produce outdated answers delivered with confidence.
How do the options compare?
A side-by-side view helps when you present the decision to others.
- Setup effort: FAQ page low; keyword search low to medium; scripted bot medium; RAG medium to high.
- Running cost: FAQ near zero; search low; scripted bot low; RAG higher, since each answer calls a model.
- Handling varied wording: FAQ poor; keyword search fair; scripted bot poor; RAG good.
- Risk of wrong answers: FAQ low; search low; scripted bot low; RAG moderate without testing.
- Maintenance: FAQ and bots need manual updates; RAG needs clean source content and re-indexing.
What does a sound RAG project look like?
Start with evaluation, not architecture. Collect 50 to 100 real questions from search logs, support tickets and sales calls, and write the correct answer for each. This set is how you will judge every version of the system.
Then build the simplest pipeline that could work: a managed option like OpenAI’s file search, Google’s Vertex AI Search, Azure AI Search or a hosted vector database such as Pinecone with a mainstream model. Measure how many test questions it answers correctly and how many it refuses when it should. Only add complexity, such as re-ranking or hybrid keyword plus vector search, when the test set shows a specific failure.
Keep a fallback. When the system is not confident, it should say so and offer a link or a human, not guess.
How Alien Road applies RAG
In a typical engagement we begin with a content and question audit: we pull site search logs, support ticket themes and the documents that should answer them. Many projects stop here, because the audit shows that rewriting ten FAQ answers solves most of the problem. When RAG is justified, we clean and tag the source documents, build the test question set with the client’s support team, and set up a managed retrieval pipeline connected to a chat widget or internal tool.
Before launch, the system must pass the agreed accuracy threshold on the test set. After launch, reporting covers answered and unanswered questions, handoffs to humans and new questions to add to the content plan. For example, an e-commerce client’s assistant might answer delivery and returns questions from current policy pages, with every answer linking to its source.
Where to start this week
Export the last three months of site search queries and support ticket subjects. Group them by topic and mark which ones your current pages already answer well. If most are covered, improve those pages first. If many questions fall between documents or use wording your content does not, you have a real case for RAG and a ready-made test set to build it against.
Keep reading: AI Optimization · AI Optimization