Building a RAG System
RAG looks simple but the details determine whether it works in production. Chunking strategy affects recall. Embedding model choice affects semantic match quality. Re-ranking separates relevant from retrieved. And RAGAS gives you four metrics to know if your pipeline is actually working.
True but incomplete
True, but it misses the point. The chunk about refunds after 14 days never reached it.
A simple RAG setup works in a demo. Real questions find its gaps.
Three places to fail
Each stage can fail: how you cut the documents, how you search, and how it answers.
Fix them in that order. The AI can only answer from what the search hands it.
How big a chunk?
Chunk size is a trade-off. Small chunks match sharply; large ones keep their context.
Exact words matter
Meaning search can miss exact codes and names. Tap the word it’s most likely to miss.
Search two ways
So many systems run keyword search and meaning search side by side, then merge the two lists.
Keyword search catches codes, names and exact phrases. Meaning search catches the same idea in other words.
A second, careful pass
Search returns twenty rough matches. For the 20-day ebook question, which should a careful re-check rank first?
Answer from sources
Tell the AI to answer only from the chunks, and to say so when they don’t cover the question.
Asking it to name the chunk behind each claim makes every answer easy to check.
Which part failed?
Test search and answers separately. Which part caused each failure?
Tap “Search” or “Answer”.
A sturdy RAG
Cut with care, search two ways, re-rank, answer only from sources, and test each part.
The terms: chunking, hybrid search, re-ranking and grounding.