RAG & Knowledge AI: Embeddings, Vector DBs & Reranking

Retrieval-Augmented Generation (RAG) enhances LLMs by retrieving relevant external documents from a knowledge base before generating responses. This eliminates stale training cutoffs and dramatically reduces hallucinations.

Documents are first split into smaller pieces via Chunking, with Chunk Overlap preserving context across boundaries. Text chunks are passed through an embedding model to produce Embeddings—high-dimensional floating-point vectors representing semantic meaning. These vectors are stored in a Vector Database (such as Pinecone, Qdrant, or pgvector) for Semantic Search using cosine similarity or dot product distance.

A Retriever fetches top-K vector matches, which are then processed by a Reranking model (e.g. Cohere Rerank) to score and reorder results by exact question relevance. Hybrid Search combines keyword (BM25) and vector search, while Graph RAG maps entity relationships across knowledge graphs for complex multi-hop reasoning.

Before

Naive Keyword Search RAG

1// ❌ Misses semantic synonyms ("cost" vs "price")2const docs = await db.query("SELECT * FROM docs WHERE text LIKE '%pricing%'");

After

Hybrid Vector Search + Reranking Pipeline

1// ✅ Combines vector embedding search with Cohere reranking2const queryVector = await embedder.embed(userQuery);3const candidates = await vectorDb.search({ vector: queryVector, limit: 20 });4const rankedDocs = await reranker.rerank({ query: userQuery, documents: candidates, topN: 5 });

Exercise

Write a document chunking utility with a 500-token chunk size and a 50-token overlap, and query pgvector using cosine distance.

Check your understanding

What is an embedding vector?Show answerHide answer
Answer
A dense numerical array (e.g. 1536 dimensions) that represents semantic meaning, where closer distances imply similar meanings.
Why is reranking important after initial vector retrieval?Show answerHide answer
Answer
Vector similarity can return broadly relevant chunks; reranking scores exact contextual relevance to select the highest-quality chunks for the LLM prompt.