The leash that kept Vector Search on topic

A marketplace’s lexical search - matching the actual words in a query against the actual words in a listing - missed an entire class of searches. “Warm jacket for winter hiking” returned nothing for an “insulated shell parka”, even though any shopper could see the results could be the same. The fix was vector search: an embedding model placing queries and products as points in a high-dimensional space, so “warm jacket” and “insulated shell” sit near each other without sharing a word. For every query, the seventy nearest product points were merged into the lexical results.

Some queries started showing no sense results. A search for “apple watch band” started returning a dog collars - no shared word anywhere between the two, a leap lexical search would never make. The embedding model was giving poor results some times.

The instinct was to make the embeddings smarter - retrain, bake category signals into the vectors. That was slow to iterate on and hard to fully debug: a bad result meant a retrain-and-hope cycle, not a cheap lever to pull.

The fix that shipped sat in front of vector search instead of inside it. The catalog already had a taxonomy - a broad category tree, and a more specific L2 subcategory beneath it. If a query’s plausible L2 categories could be guessed cheaply, the seventy vector results could be filtered down to the ones that matched, with the embeddings themselves untouched.

Most traffic didn’t need a model for that. A small number of distinct queries accounted for the bulk of searches, and each already had years of purchase history. A nightly SQL job counted, per exact query string, which L2 categories its purchases had landed in, and kept the ones above a minimum share — no training, just a lookup table.

That table only covered queries it had seen enough of. Typos, unusual phrasing, brand-new searches had no row to check. For those, a small multi-label classifier trained with FastText — light enough to run on every search — predicted plausible categories from the words themselves. It wasn’t asked to figure out exactly what the shopper meant, only which categories a match couldn’t possibly belong to: a coarse net for nonsense, not a fine one for intent.

The catch was how strict that net was set. Too strict, and it rejected a query’s own correct category along with the insane ones, whenever odd phrasing left the model unconfident. Too loose, and it let through some of the very noise it existed to catch.