Build a lexical search baseline before semantic retrieval

Exact product names and error codes provide a useful starting point for a retrieval system. Before adding embeddings, check whether a lexical baseline can reliably find the passages that contain those terms. The result gives later experiments something concrete to improve.

Create a synthetic document set containing a deployment failure code, a similarly spelled product name and an ordinary explanatory paragraph. Label the document IDs that should answer each test query. PostgreSQL full-text search can produce a first baseline without a separate search service.

SELECT id
FROM handbook_sections
WHERE to_tsvector('simple', coalesce(body, ''))
      @@ plainto_tsquery('simple', $1);

The table and parameter are application placeholders. Execute this as a parameterized query through the database driver. PostgreSQL's text-search controls describe converting documents and queries into searchable forms.

Exact code query passing through a lexical index.

Inspect the tokens you actually search

A code such as ERR_EXPORT_42 may contain punctuation or digits that a parser handles differently from ordinary words. Inspect the output of the chosen text-search parser and configuration for your exact fixtures. Do not assume that full-text search behaves like exact substring matching.

If exact code matching is important, keep a dedicated normalized error-code field with an equality lookup as another baseline. Full-text search and exact identifier lookup solve different query shapes. Record which path answered each question.

Choose a language configuration deliberately. Stemming and stop words can help natural-language questions, while names and identifiers may need preservation. Test the names users actually type, including capitalization and common spacing variants.

Compare against a labeled set

Save each query, expected source IDs and returned ordering. Include a paraphrased question that does not share the source's wording. It may expose a lexical limitation that semantic retrieval can address. Include a near-match identifier too; an irrelevant result with a similar token should not count as success.

If there are many documents, measure query plans and add an appropriate text-search index using the same configuration as the query. The excerpt above explains matching behavior and has not been executed against a database. It makes no latency claim.

When introducing a semantic retriever, keep the corpus and question set fixed. Compare which previously missed sources it finds and which exact matches it loses. A combined retrieval strategy may help, but choose it from observed failures.

A lexical baseline does not validate the eventual generated answer. Retrieval can return the correct document while the generation step misreads its exception or invents an unsupported detail.