Retrieval-augmented generation, or RAG, is a method where an AI model first searches your own documents for relevant passages and then writes its answer using them. For a small business it means an assistant that can answer questions about your services, policies or products from your real content instead of guessing. You can start with off-the-shelf tools before building anything custom.
How does RAG work?
There are two steps. Retrieval finds the most relevant pieces of your content, usually by converting text into embeddings and searching a vector index. Generation passes those pieces to a language model with instructions to answer only from them.
The approach was described in research from Meta AI in 2020 and is now the standard way to ground chatbots in company data. It is cheaper and easier to update than training a model on your documents, because changing an answer only requires editing the source text.
Does a small business actually need RAG?
Not always. If customers ask the same fifteen questions, a well-written FAQ and a rule-based chatbot may be enough. RAG becomes useful when the knowledge is wider than a FAQ can hold.
- A law or accounting firm with hundreds of guidance notes.
- A shop with a large catalogue and detailed specifications.
- A clinic or school with policies that change each term.
- A software company with long documentation and support history.
- Internal use: staff searching contracts, proposals and procedures.
What are the low-cost ways to start?
You do not need to hire engineers for a first test.
Built-in tools
OpenAI’s custom GPTs and Projects, Anthropic’s Claude Projects and Google’s NotebookLM let you upload documents and ask questions grounded in them. These are good for internal testing and for checking whether your documents are clear enough to answer from.
Website chat products
Tools such as Intercom Fin, Zendesk AI agents, Chatbase or Tidio’s AI features crawl your help pages and answer visitors with RAG behind the scenes. Pricing is usually per resolution or per message, so you pay as volume grows.
Simple custom builds
If you have a developer, frameworks such as LlamaIndex or LangChain with a vector store like pgvector or Chroma can produce a working prototype in days.
What makes RAG answers accurate?
Most failures come from the content, not the model. If two documents give different prices, the assistant may quote either one.
- Remove or archive outdated documents before indexing.
- Split long files into sections with clear headings.
- Write one fact in one place, then link to it.
- Instruct the model to say “I don’t know” and offer a human when the answer is not in the sources.
- Show the source link with each answer so users can check.
How Alien Road applies RAG for small businesses
In a typical engagement, discovery means collecting the questions customers or staff ask, usually from email, chat logs and calls, and mapping them to existing documents. Gaps become writing tasks before any technology is chosen.
Setup depends on the client’s size. For many small businesses we configure an existing chat product on the cleaned content. For clients who need control over data or custom behaviour we build a small RAG service with a vector index, a model API and logging.
Measurement uses a test set of real questions with known correct answers. We score the assistant before launch and again monthly. Reporting shows accuracy on the test set, the share of conversations handed to a human, and the top unanswered questions, which feed back into content updates.
What risks should you manage?
RAG reduces invented answers but does not remove them. The model can still misread a passage or combine two sources wrongly. Keep high-stakes topics, such as prices, legal advice or medical guidance, either in fixed answers or behind human review.
Privacy matters too. Do not index documents with personal data unless you have a legal basis and the tool’s data handling meets your obligations under GDPR or KVKK. Check whether the provider uses your data for training and how long it stores conversations.
How do you build a test set?
A test set is a list of real questions with the correct answer and the source document for each. Start with 30 to 50 questions drawn from actual customer messages, not invented ones. Include a few questions your content does not answer, so you can check that the assistant admits it does not know.
Score each answer on three points: correct, grounded in the cited source, and complete enough to be useful. Keep the scoring simple so anyone on the team can repeat it. Run the same set after every content or configuration change; a fix in one area sometimes breaks another.
Signs your content needs work
- The assistant quotes two different answers to the same question.
- Answers cite old documents you thought were removed.
- Common questions return “I don’t know” even though staff answer them daily.
Each of these points back to the source documents, which is where most improvement comes from.
Plan a monthly hour to review real conversations as well. Test sets show known questions; live logs show the new ones customers start asking after a product change or a seasonal rush.
It also helps to track which documents are cited most often. Those are the pages customers depend on, and they deserve an owner and a regular review date. Documents that are never retrieved may be unclear, badly titled or simply unnecessary, and pruning them makes the index easier to keep accurate.
Where to start this week
Gather the twenty documents that answer most customer questions, delete outdated versions, and upload them into NotebookLM or a Claude Project. Ask it the ten questions you hear most and check each answer against the source. Where it fails, fix the document, not the prompt. Once the answers hold up, you know your content is ready for a customer-facing assistant.
Keep reading: AI Optimization · AI Optimization