Hybrid search: combining keyword and vector search for RAG

Vector search is great at meaning and surprisingly bad at exact terms. Ask it about error code E1042 or a product called "Model X7 Pro" and it may return chunks that are about errors or products in general, while the one chunk containing the exact string sits on page two. Hybrid search fixes that by running keyword search and vector search side by side and merging the results. For RAG, it is one of the cheapest quality improvements you can make, and in PostgreSQL you can do it in a single query.

Why one kind of search is not enough

The two approaches fail in opposite ways:

• Keyword (lexical) search matches the words the user typed. It nails identifiers, names, codes and rare terms. It misses paraphrases: "cancel my plan" will not match a document titled "Ending your subscription".
• Vector (semantic) search matches meaning. It handles paraphrases well. It blurs exact tokens, because an embedding of "E1042" is not very different from an embedding of "E1043".

Real users mix both kinds of question, often in the same sentence. "Why does the export fail with E1042 on large files?" has a semantic part and an exact part. Hybrid search gives each part a fair chance.

Keyword search in Postgres

You don't need a separate search engine to get good keyword search. PostgreSQL full-text search handles stemming, stop words and ranking. Add a generated tsvector column and a GIN index to the same table that holds your embeddings:

ALTER TABLE chunks
  ADD COLUMN tsv tsvector
  GENERATED ALWAYS AS (to_tsvector('english', content)) STORED;

CREATE INDEX chunks_tsv ON chunks USING gin (tsv);

Because the column is generated, it stays in sync with content automatically. For queries, websearch_to_tsquery accepts the kind of input users actually type (quoted phrases, -excluded words) without throwing syntax errors.

The vector side is the pgvector setup from vector search in PostgreSQL with pgvector: an embedding column with an HNSW index and cosine distance.

Merging results with reciprocal rank fusion

Now you have two ranked lists, and their scores live on completely different scales. A ts_rank_cd value and a cosine distance cannot be added together meaningfully. You could normalise them, but normalisation is fiddly and breaks whenever the score distribution shifts.

Reciprocal rank fusion (RRF) sidesteps the problem by ignoring scores and using positions. Each document gets 1 / (k + rank) from every list it appears in, and you sum those. A document ranked first in both lists wins; a document ranked first in one and absent from the other still scores well. The constant k (60 is the common default) dampens the advantage of the very top positions.

Here is the whole thing as one query. $1 is the user's text query, $2 is its embedding.

WITH keyword AS (
  SELECT id,
         row_number() OVER (ORDER BY ts_rank_cd(tsv, q) DESC) AS rank
    FROM chunks, websearch_to_tsquery('english', $1) AS q
   WHERE tsv @@ q
   ORDER BY rank
   LIMIT 50
),
semantic AS (
  SELECT id,
         row_number() OVER (ORDER BY embedding <=> $2::vector) AS rank
    FROM chunks
   ORDER BY embedding <=> $2::vector
   LIMIT 50
)
SELECT c.id, c.source, c.content,
       COALESCE(1.0 / (60 + k.rank), 0) +
       COALESCE(1.0 / (60 + s.rank), 0) AS score
  FROM keyword k
  FULL OUTER JOIN semantic s ON s.id = k.id
  JOIN chunks c ON c.id = COALESCE(k.id, s.id)
 ORDER BY score DESC
 LIMIT 10;

Each branch uses its own index, so both stay fast. The full outer join keeps documents that only one side found. Add your tenant or permission filter inside both CTEs, never only at the end, or the limits will be filled with rows the user cannot see.

Tuning the blend

RRF works well with no tuning at all, which is why I start with it. When you have evaluation data, a few knobs are worth testing:

• Weights. Multiply one side's contribution, for example 1.5 / (60 + k.rank), if your users search for exact identifiers a lot.
• Candidate depth. 50 per side is a reasonable default. Deeper lists improve recall slightly at the cost of a bit more work.
• Language configuration. 'english' stemming is wrong for other languages. Postgres ships configurations for many languages; use 'simple' when content is mixed or full of codes.
• Re-ranking. For high-stakes retrieval, pass the top 20 fused results through a re-ranking model and keep the best five. It adds latency and cost, so measure whether it earns its place.

Whatever you change, measure it. Keep a fixed set of real questions with the known correct chunk for each, and track how often that chunk lands in the top five. The method is the same as in evaluating an LLM feature before you ship it.

When hybrid search is not the fix

Hybrid search improves recall for exact terms. It will not fix chunks that were cut in the wrong place or stuffed with navigation boilerplate. If the right passage was split across two chunks, both searches will struggle to find it. Check your chunking strategy first if results are poor across the board, and reach for hybrid search when the failures cluster around names, codes and exact phrases.

The short version

Add a generated tsvector column with a GIN index next to your pgvector embeddings, run both searches in one query, and merge them with reciprocal rank fusion. It needs no new infrastructure, no score normalisation and very little tuning, and it covers the exact-match blind spot that pure vector search leaves in most RAG systems.