Converge
Open the app

Week 11 · Season 3 · Memory

Finding things

You can build keyword, meaning and hybrid search over your own notes, add a reranker, and prove which wins with recall@k and MRR.

Open week 11 in the app

This week

  • Why search: retrieval and rankings
  • Keyword search with BM25
  • Embeddings, semantic search and vector databases
  • Scoring search: recall@k and MRR
  • Hybrid search with RRF, and rerankers

How you learn

Write BM25, cosine search, recall@k, MRR and reciprocal rank fusion from scratch in plain Python, and check each against a hand-worked example.

What you build

Point FRIDAY at a folder of your own notes. Write 20 real questions with an answer key, then build keyword search (BM25), meaning search (a free local embedding model) and hybrid search (RRF), and score all three with recall@5 and MRR.

Stretch: Add a cross-encoder reranker on the top 30, or query rewriting for vague questions, and keep it only if your scoreboard improves.

What you ship

Project 3: Ask-my-notes. Answers from your own notes with citations, "not in my notes" when it should, and a passing eval.

  • 20 questions with an answer key, from your own notes
  • search.py with keyword, meaning and hybrid search
  • A scoreboard: recall@5 and MRR for all three, in the README

Lessons

  1. 2,000 pages, one answer

    After this deck you can explain why FRIDAY must look things up before answering from your notes (retrieval), why search returns a ranking, and why only the top few results count.

    10 minFull access

  2. Count the words

    After this deck you can score pages for a question with BM25 by hand: rare words count more, repeats stop paying, and long pages get evened out. And you can say when keyword search wins.

    11 minFull access

  3. Meaning vs matching

    After this deck you can explain how embeddings let FRIDAY search by meaning (semantic search), what a vector database does, and where meaning search beats and loses to keyword search.

    11 minFull access

  4. Score the search

    After this deck you can score any search with recall@k and MRR on a question set with an answer key, and pick the number that matches how FRIDAY reads results.

    11 minFull access

  5. Best of both

    After this deck you can merge two rankings with reciprocal rank fusion (hybrid search), add a reranker to a short list, and spend a time budget where it buys the most.

    11 minFull access

  6. Under the hood of search

    After this deck you can explain how HNSW finds nearly-nearest neighbours without checking every pin, how query rewriting improves a search before it starts, and how the *k* in RRF changes who wins.

    12 minFull accessGo deeper · optional

Labs

Python in the browser, next to the lessons.

Weekend build

Skills you prove

Retrieval
Build search that finds the right passages (keywords + meaning) and cites them
Ablation
Run a controlled ablation and publish a negative result

Hand-picked official docs, papers and talks.