Production AISystems
Engineering Capability

RAG and Retrieval Engineering

Retrieval-augmented generation systems engineered for production. Ingestion and chunking, dense and sparse retrieval, query rewriting, reranking, permission-aware filtering, citations and RAG evaluation. Built to be measured on recall, faithfulness, latency and cost—not just on whether the answer sounds right.

Discuss Your RAG Architecture
Loading visual…

Engineering Decisions This Capability Addresses

Chunking strategy

Fixed-size chunks lose context. Sentence-aware or semantic chunking preserves meaning but costs more.

Hybrid vs dense retrieval

Dense captures semantics. BM25 catches exact terms. Hybrid combines both at higher latency.

Reranking depth

A cross-encoder reranker improves precision but adds 100-300ms. Depth and latency must be balanced.

Permission filtering

Filtering before ranking prevents unauthorised content. Filtering after ranking is faster but riskier.

Reference Architecture and Workflow

1

Document ingestion

Parse, chunk, embed and index with metadata and permissions

data
2

Query rewriting

Rewrite and expand the query for better retrieval

ai
3

Hybrid retrieval

Dense and sparse retrieval run in parallel

data
4

Permission filtering

Results filtered by user access rights before ranking

control
5

Reranking

Cross-encoder reranks top candidates for relevance

ai
6

Context assembly

Top passages assembled with metadata and citations

data
7

Answer generation

Grounded answer generated from retrieved context

ai
8

Faithfulness verification

Every claim checked against retrieved sources

control

Options and Trade-offs

Baseline RAG

Dense vector retrieval only. Fast, simple, but can miss exact terms.

Hybrid retrieval

Dense + BM25. Captures both semantic and exact matches. Higher latency.

Reranked RAG

Hybrid + cross-encoder reranking. Most accurate ranking. Highest latency.

Semantic caching

Cache answers for similar queries. Reduces cost and latency for repeated questions.

Evaluation, Operational Controls and Failure Handling

Recall@K

Whether the relevant passage is in the top K retrieved results

Context precision

What percentage of retrieved passages are actually relevant

Faithfulness

Whether every claim in the answer is supported by the retrieved context

P95 latency

End-to-end query-to-answer latency at the 95th percentile

Cost per query

Total cost per answered query including retrieval, reranking and generation

Solutions That Use This Capability

Enterprise AI Search

Learn More

AI Chatbot Solutions

Learn More

Conversational AI Solutions

Learn More

Discuss Your RAG Architecture

Tell us the engineering challenge you are facing. We respond with how we would approach it.

Discuss Your RAG Architecture

No finished technical specification required.

Frequently Asked Questions

When is RAG better than fine-tuning?
RAG is better when knowledge changes frequently, when answers need citations, when the knowledge base is too large for the context window or when multiple users need different answers based on their permissions. Fine-tuning is better when the model needs to learn a domain-specific style, format or reasoning pattern. They are not mutually exclusive—a combined approach can use fine-tuning for style and RAG for knowledge.
How do you prevent hallucination in RAG?
Three controls: retrieval quality (the right documents are fetched), context assembly (the model sees only relevant, permission-filtered passages) and faithfulness verification (the answer is checked against the retrieved context). If a claim is not supported by the sources, it is flagged. Evaluation sets test known queries against expected answers.
What is semantic caching and when does it help?
Semantic caching stores answers to similar queries. When a new query is semantically close to a cached one, the cached answer is returned if freshness rules allow. This reduces cost and latency for repeated questions. It is most useful when many users ask similar questions, and less useful when every query is unique.
How do you evaluate RAG quality?
Recall@K measures whether the right passage is retrieved. Context precision measures how many retrieved passages are relevant. Faithfulness measures whether the answer is supported by the sources. Answer correctness measures whether the answer is factually right. These are tracked against evaluation sets and monitored for drift over time.