Resume
Entry 3Sep 2026 · Retrieval

RAG from Scratch

  • Python
  • NumPy
  • Gemini
Question
Can I build every stage of a RAG pipeline by hand, and measure it?
Setup
Chunking, embeddings, cosine retrieval, LLM reranking and a hit@k harness, all written by hand.
Result
The reranker was silently doing nothing. Fixing its prompt contract raised hit@1 from 13/17 to 17/17.
Flow
  1. Chunk
  2. Embed
  3. Cosine top 5
  4. LLM rerank
  5. hit@k
Scope

17 questions over 6 chunks: a method check, not a benchmark.

13 → 17/17hit@1 after finding a silent reranker bug

A small set: 17 questions over 6 chunks.

Before fix13/17
After fix17/17
Fig. 3hit@1 before and after the fix. Red dots are the four the fix recovered.
Decisions
  • No framework. Lists, dicts and NumPy, so every intermediate state can be inspected.
  • Embedding calls kept apart from the similarity maths, so ranking is testable without the API.
  • The metric named honestly: hit@k, not recall@k, since each question has one answer phrase.
Run the full entryGitHub (opens in a new tab)
Entry 3, 5 of 7