The cheap side won

The recommender was asked for the twenty alternatives most likely to lead to a purchase, and the strip showed them in that order. Nothing told it to balance upsells, alternatives that cost more than the product on screen, against downsells, alternatives that cost less.

Its training data had a strong pattern: shoppers look at expensive things and buy cheaper ones. The recommender learned it well. The quickest route to a purchase was a cheaper alternative, so the downsells won. The five visible slots filled with cheaper products, often much cheaper, while the pricier alternatives sat far to the right, where few people scrolled.

Clicks and purchases from the strip went up. Then finance asked why the margin on those orders had gone down. A €120 machine bought from a €400 page is not the same sale as a €380 one, and cheaper products usually carry lower margins. The recommender had maximised purchases, exactly as asked, while pulling revenue out of the shop.

Two kinds of slot, two different jobs

The fix started with a question nobody had asked: what are the five visible slots for?

Downsells are reassurance. A shopper hesitating on a €400 machine is asking "am I overpaying?" A couple of similar machines priced just under it answer that: the price is fair, there's no hidden bargain. Many of those shoppers then buy the original. A much cheaper downsell does the opposite and sends them away with a low-margin item.

Upsells are a choice. The shopper should see something clearly better at a clearly higher price and decide whether it's worth it. Three upsells at about the same price are one choice shown three times.

Ask for more, curate the first screen

The strip now asks the recommender for thirty candidates instead of twenty, still ranked by purchase likelihood. A small selection step fills the five visible slots:

• Two downsells: of the candidates cheaper than the product, take the two most expensive, the ones closest to its own price, not the ones the recommender liked most.
• Three upsells: split the pricier candidates into three price tiers (a small step up, a medium step and a big step) and take the best-ranked candidate from each. The shopper always sees a modest upgrade, a real upgrade and a premium option.

The other fifteen places, seen only after scrolling, take the remaining candidates in the recommender's own order. A shopper who scrolls is already exploring, so there the recommender can show whatever it thinks will sell.

The recommender still decides which products are good alternatives; the selection step decides where on the price range the first five sit.

What changed

The new strip ran as a controlled experiment against the old one. Clicks dipped slightly, since the cheapest items, the most clicked, had left the first screen. Purchases from the strip held steady, margin recovered, and more shoppers who opened a downsell went back and bought the original product.

Retraining the model with margin in its objective was the obvious "proper" fix. Asking for more candidates and arranging the first screen by price got there without touching the model.

Search paid for that. A shopper typing "hand cream" got the bestseller in 30 ml, 50 ml, 75 ml and 100 ml, then its two-pack, then its gift set. Each was a perfect match; together they were one product shown six times. Other brands started on page two, and shoppers who wanted to compare never saw them.

The proper fix was to build product families: have the catalogue team group variants by hand, or train a model to cluster them. Both meant months of work, and every new product would need grouping first.

The fix that shipped never asks what a family is. It only notices when two titles look almost the same.

It starts from edit distance: the number of single-character changes (insertions, deletions, substitutions) needed to turn one title into another. "Shea Hand Cream 50 ml" becomes "Shea Hand Cream 100 ml" with two changes; turning it into another brand's cream takes many more. Dividing by the length of the longer title turns that into a similarity between 0 and 1:

$$\text{sim}(a,b) = 1 - \frac{\text{lev}(a,b)}{\max(|a|,|b|)}$$

where \(\text{lev}(a,b)\) is the edit distance between titles a and b, and \(|a|\) is the length of title $a$.

The first three pages are then rebuilt greedily, one slot at a time:

1. Retrieve everything the normal ranking puts on the first three pages, with its score.
2. Place the highest-scoring product in the next open slot.
3. Lower the score of every remaining product by how similar its title is to the one just placed, using the penalty below.
4. Go back to step 2 with the updated scores until the three pages are full.

The penalty in step 3 is:

$$s_d \leftarrow s_d \cdot \bigl(1 - \lambda \cdot \text{sim}(d,p)\bigr)$$

where \(s_d\) is product $d$'s current score, $p$ is the product just placed, and \(\lambda\) sets how hard the push is.

Because the penalties multiply, a product that resembles several already-placed items keeps sinking, while a distinct product barely moves. Variants are not removed; they give their slot to the next distinct product. Nothing leaves the three-page pool, and from page four on the normal ranking is untouched.

"Hand cream" now opened with the bestseller, then five other brands, with the bestseller's 100 ml further down the first page. Before rollout, the team reviewed the rebuilt pages for the most frequent queries with the category managers; after it, they monitored clicks beyond the first rows and immediately rephrased searches, and both moved in the right direction.

The cause was personalization. Search boosted products from the categories each shopper had been browsing, and nobody browses fridges more than someone in the middle of buying one: comparing models, rereading the same three product pages. It learned, correctly, that this shopper cared about fridges. What it never learned was that the interest ended the moment the order was placed. Past views kept pushing fridges up, whatever came next.

The obvious fix was to make personalization smarter: feed purchases into the model, retrain it to predict what each shopper wants next, until it learns that buying a fridge ends the need. That meant months of work, a new data pipeline, and choices the merchandising team could not explain. A first quick fix hid purchased items from that shopper’s results entirely. That broke a real case: shoppers searching the exact model name to find spare filters, or to buy a second unit for a holiday flat, now found nothing.

The rule that shipped sat on top of personalization instead of changing it. When a signed-in shopper has bought something from a “durable” category in the last 90 days, two things happen. The personalization boost for that category is switched off, so other fridges stop rising on past browsing. And the bought item itself has its score cut: low enough to leave the first rows, high enough to still appear when the shopper types its exact name. Everything else, including consumables, is untouched.

The list of durable categories was not written by hand. It came from order history: for each category, the share of buyers who bought again in that category within 90 days. Fridges, washing machines, sofas and mattresses sat close to zero. Coffee capsules, printer ink and water filters sat high and were left alone, since showing a past purchase there is useful. Every category below a low cut-off joined the list, refreshed monthly from new orders.

It took a few days to build, since purchase history and categories were already available at search time. After rollout, the team reviewed searches from recent buyers and monitored click-through on the first rows, which rose for that group, while “I already bought this” complaints stopped.

The trouble was that search matching is loose: a listing qualifies if it shares enough words with the query. A promoted car seat titled “baby car seat, fits most strollers” matched “baby stroller”, and could top that search. The regular results underneath were fine. The first thing a buyer saw was not.

A simple idea came from a review question: would this car seat show up for “baby stroller” if nobody had paid for it? The search engine already answered that on every request. The regular ranking was tuned for years and trusted.

So the rule became: a promoted listing may take a highlighted slot only if it also ranks high enough in the regular results. How high depends on the slot:

$$\text{organic rank limit} = 25 + 10 * \text{paid slot}$$

Paid slots are counted from zero. The top slot requires a listing in the regular top 25, the second the top 35, the third the top 45.

The slots were filled top-down. The top slot was drawn at random among promoted listings within the regular top 25, the second among the remaining ones within the top 35, and so on. A listing too weak for the top slot could still land a lower one. When no promoted listing passed the rule, the rule was dropped for that query, and slots were filled from every match as before. With nothing relevant promoted, a loose match was no worse than before, and sellers still got their visibility.

The limits were deliberately generous. The goal was never to make listings earn their place — then nobody would pay — only to require a plausible answer. Ranking 20th for “baby stroller” means the engine thinks it’s some kind of stroller. Missing the top 45 usually means it shares a word with the query and nothing more.

The car seat disappeared from “baby stroller” and kept showing for “baby car seat”, where it belonged. Reviewed against the most-searched queries, glaring mismatches were gone wherever a relevant alternative existed, promotion sales held steady, and clicks on highlighted slots went up.

For small tweaks, this was slow. Shoppers differ from each other far more than two similar rankings do, so each test took about four weeks. Eleven tweaks were waiting in line, and the proposed fix was a bigger experiment platform that would take two months to build.

Let the same shopper judge both

The team turned to interleaving AB-Test: instead of comparing two groups, it shows both rankings on the same page. For every search, both rankings are computed and a coin flip decides which one picks first. Then the two take turns, each adding its highest-placed product that is not already on the page, tagged with its team.

Take the query “tomato”. The old ranking, A, leads with fresh brandywine and cherry tomatoes. The new ranking, B, favours best-sellers, so it leads with canned chopped tomatoes and a bottle of passata. If B wins the the first coin flip and A wins the second, the page reads: canned tomatoes (B), brandywine tomatoes (A), cherry tomatoes (A), passata (B).

A shopper who converts the brandywine and cherry tomatoes gives this search to A. A shopper who converts only the passata gives it to B. Searches with equal converts, or with no converts at all, count for neither side. The coin flip keeps it fair, since top slots get more converts.

Turning wins into an answer

The headline number is the share of decided searches that B won:

$$preference = \frac{#B\ wins}{#B\ wins + #A\ wins}$$

A preference of 0.5 means no preference. To tell a real preference from luck, imagine the two rankings are equally good. Then every search would be a coin flip, and the preference would wander around 0.5 by an amount called the standard error:

$$\text{standart error} = \frac{0.5}{\sqrt{\text{decided searches}}}$$

The interval says how precise the result is. About 95% of the time, the true preference lies within two standard errors of the measured one. This range is the 95% confidence interval:

$$\text{interval} = \text{preference} \pm 2 \cdot \text{standard error}$$

The p-value says whether the result could be luck. It is the probability of getting a result at least this far from 0.5 if the two rankings were equally good. Start by measuring the distance from 0.5 in standard errors:

$$\text{distance from even} = \frac{\text{preference} - 0.5}{\text{standard error}}$$

Luck produces a distance of 1 about one time in three (a p-value of 0.32), a distance of 2 about one time in twenty (0.05), and a distance of 3 about three times in a thousand (0.003).

For “tomato”, A won 1,550 of 2,500 decided searches, so B’s preference was 0.38. The standard error was 0.01, which gives an interval of 0.36 to 0.40, 12 standard errors below even. Shoppers wanted fresh tomatoes. Across all queries, B scored 0.54 over 10,000 decided searches: an interval of 0.53 to 0.55 and a distance of 8. B won overall while losing badly on “tomato”, so the team shipped it with a fix for queries where people want the fresh product, not the processed best-seller.

The number of decided searches was fixed before each test started. Checking the p-value daily and stopping on the first good-looking day produces false winners.

Why it is faster

In an A/B test, most of the noise comes from differences between the two groups of people. In interleaving, every shopper compares both rankings on the same page. The best-seller tweak, which had needed four weeks, reached a p-value below 0.001 in three days, and the queue of eleven tweaks cleared in under a month. Replayed on six past changes with known A/B results, interleaving agreed with all six. Interleaving now screens the ideas, and only the winners go on to an A/B test.

A platform running search for a large network of independent hardware store tenants had a usual vector search problem. A query was turned by an embedding model into a point in a high-dimensional space, positioned so products with similar meaning sit nearby. “18V cordless drill” and “compact battery-powered drill/driver” landed close, sharing almost no words. The nearest points became the results.

This platform ran no category tree at all: no taxonomy to slot products into, no category list to check a result against. The embedding space’s mistakes went straight to the results page - “1/2-inch impact wrench” returning a cordless impact driver, close because both listings leaned on “impact”, “torque”, “cordless” - was typical, even though a mechanic shopping for one rarely wants the other.

What these hardware tenants did have was order history  -  useful not for products bought together, but for the opposite pattern. Feed a model every order as a sequence of products (word2vec way works) and it pulls close items that keep turning up surrounded by the same other items, whether or not they ever share an order. That’s the signature of a substitute: a shopper buying one brand’s 20V drill almost never buys a rival brand’s 20V drill too, yet the two sit in near-identical company  -  the same bits, batteries, and chargers  - across the order history. A second space, P, was built this way, alongside the search space C. Two rival cordless drills sat close in P; an impact driver, filling a different role in a job, did not.

A text query can’t be placed in P - “cordless drill” isn’t a product anyone bought - so it couldn’t re-rank results directly. What it could do was judge the result set as a group: a genuine match is mostly substitutes for one job, and substitutes cluster in P, while a stray result that made the cut in C only by shared adjectives sat apart.

So each candidate in a query’s top results got a score in P: its distance to its N nearest neighbors among the other candidates. A genuine substitute sits shoulder to shoulder with several others; a stray result sits off on its own, and that distance shows it. Scores across the result set were then compared to their own median, using the median absolute deviation (MAD) — how far a typical score strays from the median — as the unit of spread.

Press enter or click to view image in full size

$$\text{keep candidate:}\ dist_p(candidate) \leq Median + 2 \cdot MAD$$

Median and MAD don’t get dragged around by the one or two results that are wildly off, unlike a mean and a fixed cutoff would - so the threshold self-calibrated per query, nothing hand-tuned per tenant. A candidate too new or rarely bought to have a reliable score was left alone.

The list counted the last three days of searches and showed the top eight. But the most searched queries are not the most trending ones. During a July heatwave, searches for “fan” jumped eightfold and still sat in fourteenth place, behind “sofa”. Raw counts reward size. A trend is about change.

The first proposal was a forecasting model: a statistical model predicting each query’s expected demand and flagging anything well above it. For tens of thousands of queries, that was a quarter of work for a list of eight words. The real question was simpler: is this query in more demand than usual? “Usual” could be the query’s own recent past.

Step 1: count purchases, not searches

Searches alone were noisy. When a celebrity’s garden made the news, searches for “garden statue” jumped tenfold, but almost nobody bought one. So the count became converting searches: searches after which the shopper bought something from those results in the same visit.

Step 2: compare the last three days with usual

For each query, count its converting searches in the last three days (“recent”) and in the four weeks before. “Usual” is what a typical three days looked like in those four weeks:

$$baseline = \frac{\text{converting searches in the previous 28 days}}{28} \cdot 3$$

The trend compares recent demand with that:

$$trend=\frac{recent+k}{baseline+k}$$

Three days is short enough to catch a heatwave while it is still happening, and long enough for mid-sized queries to collect a useful number of purchases. Four weeks keeps one odd day from bending what counts as normal. The smoothing constant k = 10 stops a query going from zero purchases to one from scoring infinitely high; it now scores 1.1.

Step 3: rank by size and change together

The trend alone favours small queries: “hammock stand” going from 3 purchases to 25 beats “sun lounger” going from 60 to 160. Volume alone brings back the sofas. So the ranking multiplies the two:

$$score = recent \cdot trend$$

Three days in July, counting converting searches:

The heatwave rose to the top and the news story sank. But “sofa” still edges past “sun lounger” on volume alone, so two guard rails sit around the score:

• At least 20 converting searches in the three days, so a handful of buyers cannot fake a trend.
• A trend of at least 2, which keeps “sofa” and “rug” out however large they are.

Out-of-stock products cannot convert, so they never appear as dead ends.

The outcome

The list started to move. July brought fan, paddling pool and sun lounger. The team checked the list against weekly sales during the first month and monitored taps on it after rollout. Shoppers used it far more once it stopped repeating itself.

Type “cat” into a pet retailer’s search box, and the vast majority of people typing it want ads exactly on one category: “Animals > Cat”. A handful buy cat food instead, a smaller handful buy a leash or a scratching post, and a few land somewhere that fits none of the above — “Animals > Food”, “Animals > Accessories”, and “Animals > Others”, four siblings under one parent with wildly uneven popularity.

The fix was to tag the suggestion with a category chip - "cat" → "Animals > Cat”, ideally with a second chip for whichever runner-up deserved one. The question was how to pick both automatically.

The instinct was a lookup: find every past purchase that followed the query “cat”, check which category it landed in, and count. A trie can hold the query completions, indexed under “<query><separator><category id>” key and the counts as values. Walking to the prefix "cat<separator>", “category id” gave the raw count for each one in a single categories.

But a purchase in “Animals > Cat” is also a purchase in its parent, “Animals” - and so is one in “Animals > Food”, “Animals > Accessories” or “Animals > Others”. Every purchase votes for its own category and the shared ancestor above it, so “Animals,” collecting votes from all four, always ranked up the biggest raw count, even though “Animals > Cat” alone held the overwhelming majority at level 2.

The fix leaned on depth. Every category has a level — steps down from the root. “Animals” is level 1; the other four are level 2. A specific category makes a far stronger claim than the root above it, so the score needed to reward specificity instead of raw volume:

$$S(c) = N(c) \cdot x^{level(c)}$$

• N(c) is the purchase count for category c
• x is a constant greater than 1, tuned against real query logs until the picks matched what a merchandiser would call obvious.

With “Animals > Cat” already holding most of the votes, it won easily once the boost stopped “Animals” from winning on aggregate volume alone. Purchase count was the simplest choice; summing revenue or margin instead would weight a category by what it earned, not how often it sold - useful where a few high-value sales should outrank a flood of cheap ones.

The second chip was trickier. Naively, “Animals” still carried “Animals > Cat” purchases baked in alongside “Animals > Food”, “Animals > Accessories”, and “Animals > Others“ - so it won second place easily too, riding the same votes that had just crowned “Animals > Cat” the winner. That runner-up said nothing new. The fix was to make the winner’s votes stop counting once spent. After picking “Animals > Cat”, its rolled-up count was subtracted back out of “Animals” before scoring anything else:

$$N'(a) = N(a)-N(w)\ \text{for every ancestor a of winner w}$$

Recomputing after that put “Animals” back into the running on equal footing, holding only what “Animals > Food”, “Animals > Accessories”, and “Animals > Others” had contributed between them — and this time it won second place fairly. None of those three had enough conversions alone to beat the other two combined, so the honest runner-up was “Animals” itself, standing in for “something else in the pet aisle.” Only past that did “Animals > Food”, “Animals > Accessories”, and “Animals > Others” show up individually, ranked by whatever share of the votes each had earned.

The obvious fix was hardware. The team added machines to the search cluster, the group of servers that run the search engine. That doubled the hosting bill and cut the slowdown by about a third.

Looking at what people actually typed

Before the next upgrade, an engineer counted a typical evening’s queries. First they were normalized (lowercased, extra spaces trimmed) so “Black Jeans “ and “black jeans” counted as one search, then sorted by frequency. The top 10.000 queries accounted for nearly 50% of the traffic.

Answer the common questions in advance

A nightly job takes the 10,000 most common queries from the past month, runs each through the normal search engine, and stores the top 250 product IDs in a cache: a fast store where looking up a query is as cheap as reading a dictionary entry.

At search time:

1. Normalize the query.
2. If it’s in the cache, take the stored product IDs.
3. Otherwise, send it to the search engine as before.

One detail mattered most: stock. A list built at 3 a.m. can show a dress that sold out at noon, so stock is never cached. A live check keeps product only if the product is on stock. Checking stock for 250 known IDs is far cheaper than a full search, and leaves plenty to fill a page.

Searches with filters or a non-default sort skip the cache.

Time to think

The night also offered the Applied Scientists something the live path never had: time. A live search gets 10’s milliseconds; the nightly job gets hours. A heavier reranker, which re-scores the top results with a slower and more careful model, took about two seconds per query, far too slow for live traffic. Overnight it was affordable: 10,000×2sec≈5.6 hours. The busiest queries got rankings the live engine could never afford.

The result

The next evening, about half of all searches never reached the cluster. Peak load fell to daytime levels, and the slowest 5% of searches came back in about 150 milliseconds. The extra machines were removed; the cache and nightly job cost a fraction of them.

Meanwhile “garden hose”, one of the busiest spring queries, quietly stopped selling. The model’s scores looked fine. Three weeks later, a revenue report caught it. Typing the query revealed the cause in seconds: the results were connectors, nozzles and reels, with no actual hoses. A category migration had left a filter pointing at the old category. The ranking was working perfectly on the wrong products.

A relevance model judges whether the order is right. It assumes the right products are there to be ordered. The signal that did catch the problem was simpler: thousands of searches, almost no clicks.

Finding the queries that matter

The obvious report sorts queries by click-through rate, the share of searches that end in a click:

$$CTR(q)=\frac{clicks(q)}{searches(q)}$$

Sorted from the lowest CTR, it filled up with odd queries searched three times a week. Sorted by volume, it showed healthy popular queries. The version that worked estimates the clicks each query lost compared with a typical query:

$$lost(q)=searches(q) \cdot max(0, topqCTR - CTR(q))$$

Here topqCTR is the median CTR across queries with meaningful traffic. A rare query loses little, while a busy query that is slightly off loses a lot.

The hour

Every Monday, a search analyst and a merchandiser take the top 50 queries by lost clicks. They run each search live and file the cause into one of four buckets:

• Broken filter or rule: the wrong products are eligible, like the hoses.
• Bad data: missing images, a €0 price, broken titles.
• Missing product: shoppers want something the catalog doesn’t carry.
• Ranking: the right products are there, in the wrong order.

Each finding gets an owner, and the next week’s report confirms the fix: the query’s CTR recovers or it drops off the list.

Over the first quarter, ranking was the smallest bucket. “Dishwasher” had a supplier feed without images. A new phone was heavily searched two weeks before it was stocked. “Black Friday” had no landing page. None of these was something a relevance model could fix. The model project continued, now picking up its new cases from the “ranking” bucket.

Where it falls short

• It only sees busy queries. Rare searches each lose too few clicks to make the top 50, even when together they add up, and a model still has a role there. General solutions for the top queries can help in rare queries also.
• Low clicks aren’t always failure either: “opening hours” is answered on the results page, so those queries need a “known and fine” list.
• It also catches problems after shoppers have hit them, within a week instead of three, but still after.
• It only works while the hour happens.

Answering it took an analyst one day. Everything the app recorded went into a single events table, the kind a product analytics tool like PostHog or Google Analytics produces: one row per event, with an event name, a user, a session, a timestamp and a bag of properties. A search, the click on a result, the basket addition and the order were four different rows, sometimes hours apart, among millions of page views and button taps. Linking an order back to a search meant joining that table to itself over and over, digging item IDs out of the properties, lining events up by time, then making a call on the awkward cases. If a shopper searched “pillow”, then “memory foam pillow”, and bought from the second results page, which search gets the credit? What if the purchase came three hours later?

A month later, a second analyst made different calls and got a different number. Neither was wrong, but they couldn’t both be reported.

Every analysis was deciding attribution, which search gets credit for which action, from scratch. The fix was to decide it once, write the answer down, and let everything else read from that.

The table

The search attribution table is built from that same events table once a day, overnight, when the previous day’s orders have mostly come in. It has one row for every search request, with fields like:

• request_id, user_id, session_id, search_ts
• query and filters, exactly as the shopper sent them
• results: the list of items shown
• max_level: the furthest the shopper got with any of those results

Each entry in results holds the item_id, its position on the page, its category at three depths (category_l1 like Bedding, category_l2 like Pillows, category_l3 like Memory Foam Pillows), a level, and the features: the numbers the ranking system used to order it at the moment of the search.

The level records what happened to that item after this search:

• 0: shown, no interaction
• 1: clicked
• 2: added to basket
• 3: purchased

Other businesses use different steps, but higher always means further down the funnel. The request’s max_level is just the best outcome on the page:

$$max_level = \max_{i \in results} level_i$$

The nightly job holds the rule that used to live in each analyst’s head. A simple version credits an action on an item to the most recent search in the same session that showed it. The exact rule matters less than having exactly one.

The questions become short

How well does each query do? A search “gets a click” if anything on the page got a click, so the funnel rates for a query q are:

$$rate_k(q)=\frac{# searches\ of\ q\ with\ max_level \geq k}{#searches\ of\ q}$$

SELECT 
  query,
  COUNT(*) AS searches,
  AVG(CASE WHEN max_level >= 1 THEN 1.0 ELSE 0 END) AS click_rate,
  AVG(CASE WHEN max_level >= 2 THEN 1.0 ELSE 0 END) AS basket_rate,
  AVG(CASE WHEN max_level >= 3 THEN 1.0 ELSE 0 END) AS purchase_rate
FROM search_attribution
WHERE search_date >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY query
HAVING COUNT(*) >= 100
ORDER BY searches DESC;

The merchandiser’s one-day question became one row of this output.

How much does position matter? Expanding the results list into one row per item makes it possible to ask how often each slot gets clicked, whatever is in it:

SELECT 
  r.position,
  AVG(CASE WHEN r.level >= 1 THEN 1.0 ELSE 0 END) AS ctr
FROM 
  search_attribution s
  CROSS JOIN UNNEST(s.results) AS r
WHERE s.search_date >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY r.position
ORDER BY r.position;

(The syntax for expanding a list varies by database: UNNEST, LATERAL VIEW explode, and so on. The idea doesn't.)

Which categories do people actually want for a query? This is the probability that a search for q leads to an interaction with at least one item in category c:

$$P(c|q)=\frac{# searches\ of\ q\ with\ an\ interaction\ in\ c}{#searches\ of\ q}$$

WITH 
  hits AS (
    SELECT
      s.query, 
      r.category_l2,
      COUNT(DISTINCT s.request_id) AS cnt
    FROM 
      search_attribution s
      CROSS JOIN UNNEST(s.results) AS r
    WHERE 
      s.search_date >= CURRENT_DATE - INTERVAL '30' DAY
      AND s.max_level >= 1
      AND r.level >= 1
    GROUP BY ALL
  ),

  totals AS (
    SELECT 
      query, 
      SUM(cnt) AS total_cnt
    FROM hits
    GROUP BY query
  )

SELECT 
  h.query, 
  h.category_l2,
  cnt / total_cnt AS p_interaction
FROM 
  hits h
  JOIN totals t ON t.query = h.query
WHERE t.total_cnt >= 100
ORDER BY h.query, p_interaction DESC;

For “pillow”, Pillows came out far ahead, with Outdoor Cushions a distant second. That’s exactly what a query-to-category classifier (a model that guesses which category a query belongs to) is trained to predict. For busy queries it can simply be counted, and used to boost the categories shoppers actually engage with.

The side effect nobody planned for

Each result stores its features, as they were at search time, next to what the shopper did. That makes the table a ready-made training set for a ranking model, with features that match what the live system saw.

The limits

Buying on a phone after searching on a laptop, or coming back the next day, falls outside a session-based rule. Purchases after the nightly cut-off are missed until a later rebuild, so the latest day always looks a little worse than it was. And it’s a day behind by design: it answers “what happened yesterday”, not “what’s breaking right now”.

The CEO queue told the story better than any dashboard. “Searched ‘fridge,’ got zero results. Typed ‘refrigerator’ instead, worked fine”. A new product line launched with a slang name the catalog didn’t use anywhere, and searches for it just died. Every fix was the same: find the file, add a line, get it reviewed, redeploy. For a one-word problem, that’s an absurd amount of ceremony - and the list only ever grew, because nobody wanted to be the one who deleted an old line and broke something.

What changed my mind was a pattern in the session logs - the record of what one visitor searched, in order, during one visit. People search “fridge”, see nothing worth clicking, and ten seconds later, same session, search “refrigerator” and click something. They’d already solved my problem for me.

So instead of waiting for a human to notice, I started mining for that pattern: pairs of searches, same session, where the first got no clicks and the second - worded differently - got one. Frequency does the filtering. One person typing “fridge” then “refrigerator” is a coincidence. A few people doing it in a month is a synonym.

Except it’s noisier than that sounds. A month of mining produces thousands of candidates, and many aren’t synonyms: typos (“resturant” → “restaurant”), plurals, unrelated searches that landed in the same session, and sneaky near-misses where the second query narrows the first rather than restating it — “sofa” then “sofa bed” isn’t the same intent, it’s a more specific one, and treating it as a synonym would quietly break search for everyone.

Before anything gets near a human — or an AI — I run two cheap filters. First, repetition: drop any pair that didn’t recur at least a handful of times, since a one-off is noise, not a pattern. Second, edit similarity — how many single-character changes it takes to turn one string into the other. “resturant” and “restaurant” are one typo apart; “fridge” and “refrigerator” aren’t close by that measure at all. Requiring low edit similarity strips out most typos and plurals before they cost a single review cycle.

What survives is noisy in the interesting way - real reformulations, including intent-narrowing ones like “sofa bed” that no string-distance check catches. That batch goes to an LLM: pair, frequency, a few example queries, one narrow yes/no question - same intent, or different? Not inventing synonyms, just judging ones behavior already nominated. I still spot-check a sample before anything ships, because it’s caught waving through a few of those near-misses too.

It’s not a complete fix. Brand-new terminology has no reformulation history yet, so this catches nothing until people have already been frustrated by it once. It only sees users who retry - the ones who search once, get nothing, and leave never show up in a log pair at all. For anything new or high-stakes, I still keep a thin manual layer around. But for the long, unglamorous tail of synonyms, this replaced a dictionary that decayed the moment I stopped staring at it with one that keeps itself roughly current on its own.

Why the Usual Fixes Fall Short

Fuzzy matching against the catalog. The fuzziness parameter lets queries match product titles within an edit distance of 1 or 2. It’s easy to enable and catches obvious slips. But it optimises against the wrong corpus. Product titles are written by merchandisers, not shoppers — full of brand names, model numbers, and marketing phrasing that customers rarely type. “Sweter” may resolve to “sweater” or “sweeter”, won’t reliably resolve to the right item just because some listing happens to contain a nearby string; the catalog wasn’t built to model how people search, so fuzzy matches against it are essentially guesses dressed up as corrections.

The built-in suggester (e.g., phrase or term suggesters). These generate candidate corrections from the indexed terms themselves - meaning they inherit the same catalog-language bias, plus they tend to suggest whatever term is most frequent in the index, not whatever term shoppers actually intended.

Generic spelling dictionaries. Off-the-shelf dictionaries (aspell-style corpora, general English word lists) know that “recieve” should be “receive”, but they have no concept of product vocabulary, brand names, or category-specific slang. They’ll “correct” a legitimate product term into an unrelated dictionary word, or fail silently on domain-specific misspellings entirely.

The common thread: all three approaches try to infer intent from data that doesn’t represent the customer’s language. What’s missing is a model of how people actually search — which is sitting in your query logs the whole time.

Correct against queries, not products

A more reliable approach flips the source of truth from the catalog to real search behavior:

1. Select top-converting queries from logs. Pull queries that reliably lead to clicks, add-to-carts, or purchases — this is your “known good” query vocabulary.
2. Index those queries as keyword. Store them in a lightweight side index, separate from the product catalog, preserving exact strings.
3. On low or zero results, search this query index for low edit-distance matches and re-run the user’s search using the corrected query instead of the catalog.

The precision gain comes from matching against the right vocabulary.

A few examples

The german typo “wane” is matching “wand” (wall) instead of the likely more frequent “wanne” (tub):

But Germans are not alone, consider some english ambiguous cases:

• “sweter” is edit-distance 1 from both sweater and sweeter.
• “blendr” is edit-distance 1 from both blender and blander.
• “toastr” is edit-distance 1 from both toaster and toastier.

A catalog or dictionary has no way to break these ties — both candidates are equally plausible strings. But when you match against top-converting queries, the ambiguity mostly resolves itself: “sweater” appears constantly as a converting search term, while “sweeter” essentially never does, because nobody searches an e-commerce site for the word “sweeter” on its own. The frequency and conversion signal embedded in real query logs acts as a prior that pure string-distance math can’t provide.

This also means the correction candidate pool shrinks to queries people actually type with purchase intent, so a low-frequency edit-distance neighbor doesn’t get promoted over the obviously correct, converting query. The system isn’t just “spelling aware” — it’s shaped by demand.

On cases where no similar popular query is found, a catalog fallback can help.

A home goods retailer’s search team got the same complaint from two different directions in the same week. A shopper wrote in asking why a rug she’d bookmarked, out-of-stock since Tuesday, had simply disappeared from search — she’d wanted to sign up for a restock alert, and now the listing was gone, as if it had never existed. Two days later, a merchandiser flagged the opposite problem: a lamp that had been sold out for six weeks was still turning up on the first page for “floor lamp”, pushing aside things shoppers could actually buy.

Both complaints traced back to the same rule. The search index treated stock as a switch: in stock, the item ranked normally; out of stock, it dropped out of results the moment inventory hit zero. That handled the obvious case - nobody wants a page full of things they can’t buy - but it was too blunt for what shoppers and the business both needed. An item that sold out an hour ago is usually still worth showing: it might restock by the weekend, and someone browsing “floor lamp” today may well still be shopping next week. An item that’s been gone for two months is a different story. Same rule, wildly different situations.

The team’s first instinct was to patch the threshold: keep out-of-stock items visible for exactly three days, then drop them. It fixed the rug case and, for a while, looked like enough. But three days is just as arbitrary as zero. A popular item worth featuring for a week got cut off early; a niche item nobody would look for again lingered exactly as long as the bestseller next to it. The threshold didn’t know the difference between “temporarily gone” and “gone for good” - it only knew how to count to three.

Someone on the team then floated training a model: feed in category, past restock frequency, how fast the item used to sell, and let it learn how long an out-of-stock listing should stay visible before it’s more clutter than help. It was shelved almost as fast as it was proposed. Restock timing is noisy - a supplier delay or a shipping snag can throw off any pattern the model had learned - and nobody wanted to explain to a merchandiser why the model kept a slow-moving lamp visible for eleven days but pulled a fast-selling one after four, especially when the honest answer would be “the training data said so.”

What shipped instead borrowed an idea from how audio fades out instead of cutting off: rather than a hard cutoff, out-of-stock items lose a share of their search score every day they stay unavailable, following a smooth curve instead of a step function. The formula reused the same score the item already had in stock, and applied a decay factor based on how many days it had been out:

$$score_{dacayed}=score_{in_stock} \cdot 0.5^{\frac{days_out_of_stock}{half_life}}$$

With “half-life”, the number of days it takes for the score to drop by half. With a half-life of five days, an item that sold out yesterday still ranks close to where it did in stock; by day five it’s ranked at half its normal strength; by day fifteen, an eighth. Nothing vanishes on a fixed date. A rug gone for a day barely moves. A lamp gone for two months has quietly decayed to a score so low it never surfaces on a normal search, without ever being explicitly removed - so a shopper who searches by exact name, or has it bookmarked, can still find it and sign up for a restock alert.

The half-life itself didn’t need to be one number for the whole catalog. Fast-moving categories like phone accessories, which restock quickly or get replaced by a near-identical model, used a shorter half-life; slower categories like furniture, where a “back in three weeks” is common and worth waiting for, used a longer one. Picking those numbers per category was still a judgment call, made by people who knew the category, not a model — but it was one knob per category instead of one for the whole catalog, and easy enough to explain in a sentence: “rugs fade out over three weeks, phone cases over four days.”

Some queries started showing no sense results. A search for “apple watch band” started returning a dog collars - no shared word anywhere between the two, a leap lexical search would never make. The embedding model was giving poor results some times.

The instinct was to make the embeddings smarter - retrain, bake category signals into the vectors. That was slow to iterate on and hard to fully debug: a bad result meant a retrain-and-hope cycle, not a cheap lever to pull.

The fix that shipped sat in front of vector search instead of inside it. The catalog already had a taxonomy - a broad category tree, and a more specific L2 subcategory beneath it. If a query’s plausible L2 categories could be guessed cheaply, the seventy vector results could be filtered down to the ones that matched, with the embeddings themselves untouched.

Most traffic didn’t need a model for that. A small number of distinct queries accounted for the bulk of searches, and each already had years of purchase history. A nightly SQL job counted, per exact query string, which L2 categories its purchases had landed in, and kept the ones above a minimum share — no training, just a lookup table.

That table only covered queries it had seen enough of. Typos, unusual phrasing, brand-new searches had no row to check. For those, a small multi-label classifier trained with FastText — light enough to run on every search — predicted plausible categories from the words themselves. It wasn’t asked to figure out exactly what the shopper meant, only which categories a match couldn’t possibly belong to: a coarse net for nonsense, not a fine one for intent.

The catch was how strict that net was set. Too strict, and it rejected a query’s own correct category along with the insane ones, whenever odd phrasing left the model unconfident. Too loose, and it let through some of the very noise it existed to catch.

Two months before Christmas, a Category Manager flagged a familiar worry: a run of seasonal products  - ornaments, advent calendars, holiday sweaters -  had to sell through by December 24. After that date, the same stock would sit in a warehouse for eleven months, worth a fraction of its current price. The catalog held thousands of items, but only about 5% of them carried this deadline. Everything else was business as usual. The lever here was search ranking: nudge these items higher while there was still time to move them, without disturbing anything else.

The starting point was a number the search team already had on hand from an earlier stock-based ranking project: expected_days_of_stock, how many days current inventory would last at the item’s average sales pace.

$$expected_days_of_stock=\frac{total_stock}{average_saily_sales}$$

​That number had previously been used to reward comfortable runway - items unlikely to run out soon got a small lift. Here the need was the opposite: flag items that wouldn’t sell through in time. Dividing expected_days_of_stock by the shopping days remaining until the deadline gave a risk ratio.

$$risk_ratio=\frac{expected_days_of_stock}{shopping_days_left}$$

The first attempt at shopping_days_left simply counted the calendar days to December 24. That overstated how much time was actually left: an order only helps if it arrives in time to go under the tree, and delivery itself takes days the customer doesn’t get to spend shopping. shopping_days_left had to be the days until the deadline minus the item’s expected delivery time, not the raw calendar gap.

$$shopping_days_left=days_until_deadline - expected_delivery_days$$

A slow-shipping item - bulky, or fulfilled from a distant warehouse - effectively ran out of runway earlier than a fast-shipping one with identical stock and sales pace, and the ranking needed to reflect that instead of treating every item as if it could arrive overnight.

A risk_ratio > 1 meant the item was on pace to still have stock left after its last realistic ship date — exactly the case that needed a push.

The first draft applied risk_ratio catalog-wide. That was wrong on its face: for 95% of the catalog there was no December 24 deadline at all, so shopping_days_left had no real meaning, and the formula would have produced boosts (or penalties) for items that were never part of the problem. The fix wasn’t a tweak to the math — it was a scope flag, computed upstream, that zeroed the boost for anything outside the seasonal set before the formula ever ran. The formula itself was never trusted to stay neutral on its own; scope was enforced separately.

For the 5% actually in scope, a plain linear boost above risk_ratio = 1 behaved badly at both ends. Just past the threshold, the boost was too small to matter - an item barely over the line got barely more visibility, when it needed a push soon. Far past the threshold, a straight line kept climbing without bound, letting a handful of badly overstocked items dominate results regardless of relevance. What was needed was a curve that responded quickly near the threshold and then leveled off - the same shape as a cup of coffee cooling toward room temperature, fast at first, then flattening out. That’s a half-life decay, capped by a constant K that sets the maximum possible boost:

$$boost=K\cdot(1-2^{- \frac{risk_ratio-1}{half_life}})$$

At risk_ratio = 1, the exponent is zero and boost = 0 - no bonus yet. As risk_ratio climbs past 1, boost rises quickly at first and then approaches K without ever quite reaching it. half_life is the amount by which risk_ratio must exceed 1 for the item to earn exactly half of K - the same halving-distance idea as a physical half-life, just running toward a ceiling instead of toward zero.

As shopping_days_left shrank on the way to December 24, half_life shrank with it in lockstep, so the same amount of excess risk saturated the boost faster automatically as the deadline approached - no manual retuning, and no separate parameter to tune per item or category.

The boost was folded into ranking multiplicatively, not additively:

$$final_score=base_relevance \cdot (1+boost)$$

This kept the risk boost a nudge, not an override - a seasonal item with weak relevance to the query still couldn’t outrank a strong match on a competing item; it only closed the gap between comparably relevant results.

The one place this needed a hard rule, not just a formula, was the deadline’s final approach. As shopping_days_left approaches zero, the dynamically shrinking half-life shrinks with it, and the boost curve stops looking like a smooth ramp and starts looking like a step function - full boost or none, with almost no room in between. Shopping_days_left was floored at 1 to keep the curve from degenerating entirely, but that’s a patch on the edge case, not a fix for it: the closer to the deadline, the less gracefully this technique behaves, which is worth knowing before leaning on it in the final days of a sale.

An online retailer’s search box had a problem that only showed up in demos nobody meant to give. Type “kids bi”, and the dropdown confidently suggested “kids bicycle helmet flame retardant”. Nothing on the site matched. The suggestion wasn’t wrong exactly — grammatical, on-topic, plausible — it was just invented.

The autocomplete ran on a sequence-to-sequence model, the architecture family used for machine translation, trained to predict how a partial query continues. Fit on historical query logs, this seemed reasonable: generalize to prefixes it hadn’t seen verbatim. In practice, “generalize” and “invent” were the same behavior wearing different clothes — the model returned things that sounded like what people had searched for, not things they actually had. Nobody could tell from a single suggestion; it took a customer hitting zero results to notice.

The fix was almost insultingly simple: a trie. A trie is a tree where each path from the root spells out a prefix, and any node can be marked as the end of something real — a standard first-year data structure, usually taught for spell-checkers.

The trie was built from the query log, restricted to phrases that had actually led to a purchase, each carrying a weight for how often it converted. Typing “kids bi” walked the trie to that prefix and returned the completions with the highest weight underneath it. If nobody had ever bought anything down the “flame retardant” branch, no such suggestion could exist. The structure can’t hallucinate; it only returns paths that were put there.

The weighting mattered as much as the trie itself. An earlier version weighted by raw query frequency, surfacing typos ahead of the correct spelling, since misspellings compound quietly. Weighting by conversions meant a completion had to be popular with people who actually finished the task — and it was easy to defend internally, since anyone could read the trie and see why a suggestion appeared, no vectors or attention weights to interpret. Debugging became a lookup, not an investigation.

The catch: a trie only knows what’s already happened. It offers nothing for a genuinely new prefix until purchase history accumulates. The model’s worst flaw, inventing nonsense, was also its only advantage: it could say something about a prefix it had never seen. The trie says nothing at all, which is safer, but cold-start queries still need a fallback.

A popular sofa had five units left in the warehouse. By any normal accounting, that’s “in stock” — the listing page said so, the relevance model gave it full weight, and it sat near the top of search results for “grey sofa.” What the ranking didn’t know was that this particular sofa was selling fifteen units a day. Five units of inventory against that pace is closer to “gone by tonight” than “available.”

The first instinct was to work with the number already sitting in the product database: total units on hand. Penalize anything below some threshold, say fewer than five in stock. It didn’t hold up. Five units of that fast-selling sofa was basically already gone. Five units of a slow-moving accent chair that sells twice a week was weeks of comfortable buffer. Same number, opposite urgency — raw unit count says nothing about how fast those units disappear, and any threshold drawn on that number is really a guess about a category’s typical pace, dressed up as a fact about one item.

What the ranking actually needed wasn’t a count of units. It was a measure of time: given how fast an item is currently selling, how many days would its current stock last? Call that expected_days_of_stock. A sofa with five units and heavy daily demand might have less than a day of runway left, even while the stock count alone still reads "available." An accent chair with the same five units and light demand might have weeks. Once stock is expressed in days instead of units, it becomes directly comparable across an entire catalog, regardless of how fast or slow any individual category normally moves.

With that number available per item, the boost itself is a single ratio, folded into the existing score:

$$final_score = base_relevance \cdot boost$$

$$boost = 1 + \frac{K \cdot expected_days_of_stock}{expected_days_of_stock + half_life}$$

At expected_days_of_stock = 0 — no runway left — the fraction is zero and boost = 1: no bonus, the item ranks on relevance alone. As expected_days_of_stock grows, the fraction climbs toward 1 and boost climbs toward 1 + K: a comfortably-stocked item gets up to a K-sized lift over an otherwise identical item that's about to run out. K sets the ceiling on how much stock can matter relative to relevance — a small K means stock only breaks close ties; a larger one lets it move items further. Framed as a boost for abundance rather than a penalty for scarcity, an item never gets punished below its relevance score for being low on stock; it simply stops earning the bonus that better-stocked competitors get.

half_life is what keeps that curve from being a step function. It's the number of expected days of stock at which an item has earned exactly half the maximum boost — the point where the fraction equals 0.5. Below it, gains come fast: going from half a day of runway to two days moves the needle a lot. Above it, gains taper off — going from thirty days to sixty barely changes the boost, because thirty days was already comfortably safe. One knob, tuned per category to whatever "comfortably stocked" means for how fast that category typically sells.

Computing expected_days_of_stock itself comes down to dividing stock by a sales rate:

$$expected_days_of_stock = \frac{total_stock}{a \cdot M + (1 - a) \cdot N}, 0 < a < 1$$

M is the item's own average daily sales — the most direct signal, when there's enough history behind it. N is the average daily sales among the category's top sellers, standing in for "how fast an item like this typically moves." a weights how much to trust the item's own number against that category benchmark. A brand-new listing has no real M to speak of — zero or one lucky sale tells you almost nothing — so leaning on N avoids treating an unproven item as either doomed or immortal. An item with months of steady sales history has an M worth trusting on its own.

That argues for a itself moving with how much history an item actually has. A simple version: count the number of days in the last 30 that the item was available for sale at all, and scale a by that fraction —

$$a = \frac{days_available_last_30}{30}$$

A listing available for all 30 days gets a = 1: its own sales history fully determines its rate, no category blending needed. A listing available for 3 of the last 30 days gets a = 0.1: almost entirely deferring to the category benchmark, since three days of data barely qualifies as a track record yet. As the item accumulates more selling days, a rises on its own, and the formula gradually shifts its trust from the category's average to the item's own proven pace — no separate rule needed for "new" versus "established" items, just the same fraction recalculated as history accumulates.

It’s still an estimate, not a guarantee. M and N are both historical averages, and a sudden spike — a listing that goes viral overnight — can burn through stock far faster than its recent history predicts, so the formula reacts a beat behind reality rather than ahead of it. half_life is a good feature for category responsible to manage. K is still tuned by hand but it can be learned. A formula built from sales counts and stock counts the system already tracked did more to keep search results buyable than the demand-forecasting model that would have been the "proper" way to solve it.

The first attempt at a fix was the obvious one: a flat freshness multiplier. Anything listed today got a 3x boost. A month old, the multiplier faded to 1x, no boost at all. Simple, cheap, easy to explain to anyone on the team.

It broke almost immediately. A shopper searching for a specific model number — a rare, exact match, the kind of listing that should win every time — got outranked by a brand-new listing that barely matched the query at all. The multiplier didn’t care how strong the underlying match was; it just multiplied whatever score walked in the door. A weak match times 3 could easily beat a strong match times 1. Freshness wasn’t adding a nudge, it was overriding relevance entirely.

The real fix needed two properties the flat multiplier didn’t have: a decay that felt natural instead of a cliff, and a ceiling that kept freshness from ever overpowering a genuinely better match.

The decay came from a half-life curve, the same shape used to describe radioactive decay or a cooling cup of coffee:

$$\text{freshness(age)} = 0.5 ^ {\frac{\text{age}}{\text{half life}}}$$

A listing at exactly one half-life old is worth half its original freshness; two half-lives, a quarter; and so on. No arbitrary cutoff, just a smooth slope down.

The ceiling came from capping how much that freshness score could move the needle:

$$\text{final_score} = \text{base_relevance} \cdot (1 + \text{max_boost} \cdot \text{freshness(age)})$$

With max_boost set at 0.15, a brand-new listing can picks a 15% bump. That's enough to pull a comparable new listing ahead of a stale one, but nowhere near enough to let a weak match leapfrog the exact model-number hit. Relevance stays the senior partner; freshness just gets a vote, not a veto.

Two objections came up before this shipped. First: doesn’t a decay curve still let a rotating cast of new arrivals crowd out listings that have proven themselves over time? It does, if freshness is the only signal in play. The fix was to blend the freshness boost with a click-based performance score, so a listing that’s been converting well for weeks keeps its standing on merit, while new listings only get their temporary lift on top of that, not instead of it.

Second: is calendar time even the right way to measure “new”? A listing posted at 2 a.m. and one posted at peak traffic aren’t actually the same age in any way that matters — what matters is whether the item has had a fair shot at being seen. That reframed the whole formula around exposure instead of the clock:

$$\text{freshness(impressions)} = 0.5 ^ {\frac{\text{impressions}}{\text{half life impressions)}}}$$

Same shape as before, just decaying by impression count rather than elapsed hours — half_life_impressions is the number of views after which a listing's freshness has faded to half its starting value. An unlucky listing with low traffic no longer gets penalized for time it never got to prove itself in.

None of this makes freshness a solved problem. The half-life and the cap are still tuned by hand, and what counts as “enough” impressions before a listing ages out of its boost is a judgment call, not a law of nature. But it beats the alternative — a multiplier that couldn’t tell a rare exact match from noise.

The Standard Fix: Decompounders

The usual approach in OpenSearch is a dictionary_decompounder. You feed the analyzer a word list, and at index time it checks whether any word on that list is a valid subword of the compound. Tischlampe splits into Tisch and Lampe, both get indexed, and a query for either one matches.

This works until the word list produces nonsense splits. The classic case: britisch (british),  fantastisch (fantastic) and asiatisch (Asian) — neither has anything to do with furniture, but if Tisch is on the word list, the decompounder strips “-tisch” off both and injects a spurious Tisch token into unrelated documents. Now “Tisch” returns furniture, adjectives, and Asian restaurants.

The Other Direction: Merging Words Back Into Compounds

Decompounding solves one half of the problem: a single word matching a compound. The opposite failure happens too — “tisch lampe” should also match a document containing tischlampe. The fix is a synonym contraction rule, mapping a multi-word input to several output tokens:

tisch lampe => tischlampe, tisch, lampe

Read left to right: the phrase expands to the compound token plus each original word, preserving recall even if only one word appears in the index.

Picking Candidates Before You Even Call an LLM

Before any rule gets generated, you need a candidate list — and most of the noise can be filtered out here, cheaply, without an LLM.

Product data. Mine the most frequent words in product titles and attributes, keep only those above a minimum length.

Converting search queries. Product-data frequency tells you what’s in your catalog; query-log frequency tells you what customers type. Check top converting queries for compounds your index can’t decompose.

Using an LLM to Build the Rules

Prompt the model with the compound word (batched) and ask it to return the components. LLM writes rules like:

tischuhr => tisch, uhr

That decomposition alone isn’t the index-time rule. A conversion step — plain code — turns it into the merged form: the parts space-separated on the left, the compound plus parts as output:

tisch uhr => tischuhr, tisch, uhr 

Batched across candidates, this gives you:

tisch uhr => tischuhr, tisch, uhr 
boden lampe => bodenlampe, boden, lampe
nacht tisch => nachttisch, nacht, tisch

This works for genuine compounds, but the LLM over-generates for the same reason a naive lookup table does — it’s pattern-matching on morphology, not taxonomy. At this stage every rule keeps both parts unconditionally, since pass one can’t know tisch isn’t the right category for a table clock.

Second Pass: Generalize, Then Validate

Pass two takes the compound and checks each fallback token against the category tree, dropping whatever doesn’t belong:

tischuhr => tischuhr, uhr 
bodenlampe => bodenlampe, lampe 
nachttisch => nachttisch, tisch

Tischuhr keeps uhr, drops tisch — it’s a clock, not a table. Bodenlampe drops boden likewise. Nachttisch keeps tisch, since a nightstand genuinely is a table. Only survivors make it into the live dictionary.

Doing It All at Index Time

When content is written once and read many times — an article, a product description, any user-generated text — both passes belong at index time. Do the work once, at write time, and every search stays a plain match.

Pass two is a single-token input, the safe case for a synonym filter: put it in the index analyzer, and Tischlampe tokenizes as [tischlampe, lampe] at write time — tisch never enters the stream. This replaces the naive dictionary_decompounder outright, using the validated dictionary instead of a flat word list.

Pass one works the same way: if a writer types “tisch lampe”, indexing it injects tischlampe, tisch, and lampe into that document's token stream at write time — every future query for any of the three already matches.

With both passes at index time, the query side is trivial:

• lampe → matches the lampe token.
• tischlampe → matches the tischlampe token.
• tisch lampe (however the customer split it) → matches on tokens already present.

synonym vs. synonym_graph. The legacy synonym filter never produces a token graph, so it drops straight into an index analyzer — but it mishandles position increments when input/output token counts differ, exactly the case here. Plain match is usually fine; phrase queries and highlighting can come out wrong. synonym_graph represents the relationship correctly, but Lucene's inverted index can only store a linear position sequence, so a graph output must be flattened first:

synonym_graph → flatten_graph

This collapses the graph into a position-safe stream. The trade-off: it fixes matching but not exact phrase fidelity — if the field also needs phrase queries or highlighting, keep a second, lightly-analyzed field for that.

Why This Matters: The furniture Example

You can see the naive version of this failure live on this furniture retailer. Search for “tisch” (table) and, alongside actual tables, the results include two Tischlampe (table lamp) and one Tischuhr (table clock) on the top six results. Both got indexed as Tisch because they contain the string, not because a customer looking for a table wants a clock or a lamp.

It’s a small bug, but it’s exactly the kind of relevance issue that erodes trust in a search bar — and exactly the kind of thing a taxonomy-aware LLM pass is well suited to catch before it ships.