Embeddings for ML system design interviews 9 questions in 4 chapters

A one-hot ID gives each item its own weight, so nothing learned about one item helps a similar one, and new items have no slot. An embedding is a short dense vector learned so that items that behave alike sit close together. What close means comes entirely from the training objective: bought together suits upsell, while similar descriptions suit search.

Put past orders in a cart-by-product matrix and learn two thin matrices whose row dot products reconstruct it. Purchases pull a cart and its products together, while empty cells push weakly, because not buying is weak evidence. The product rows become item vectors. It is cheap and well understood, so it is a strong baseline that proves later complexity is worth it.

One tower encodes the query side, such as the cart, and one encodes candidate products into the same space, so their dot product is the score. I hold one item out of each completed order as the positive, use other items in the batch as negatives, and train with InfoNCE. I add periodically mined hard negatives, correct for popularity bias and split evaluation by time.

A graph helps when links such as product to brand, category or promoter carry signal that carts miss. node2vec turns random walks into sentences and trains a word2vec-style objective, but learns one vector per node, so new nodes wait for retraining. GraphSAGE learns a function over a node's features and sampled neighbours, so a new product gets a vector immediately. Always justify the graph first.

A store catalog is far too small to pre-train a large language encoder, so I start from a pre-trained one. I size it to where it runs: catalog items are embedded in batch, so a larger model is affordable, while queries are embedded per request, so latency pushes toward smaller. I fine-tune contrastively on my own pairs, or use LoRA adapters, and weigh hosted APIs against cost and deprecation risk.

Embeddings are used in three ways. As features, they replace one-hot IDs and transfer knowledge to other models. For clustering, items close together form segments or near-duplicate groups. For retrieval, I embed candidates offline into an approximate nearest-neighbour index, embed the query online, and pass the neighbours to a heavier ranker. Text search usually runs hybrid with keywords, and RAG puts retrieved documents into a prompt.

I check intrinsic health first: a histogram of similarities on held-out data quickly shows a collapsed space where everything looks similar. Then I measure where the embedding is used. For retrieval that is recall at k on held-out, time-split orders, confirmed by an online test of the product metric. For a feature, I run an ablation: train the downstream model with and without it on the same data.

More dimensions capture more nuance but cost more to train, store and search, so I size by whether serving cost or quality dominates. Matryoshka training lets one vector's prefixes work alone, so a short prefix retrieves and the full vector reranks. Retraining creates an incompatible space, so I re-embed the corpus into a new index and switch atomically. Freshness needs batch re-embedding plus event-driven updates.

Content features such as title, category and price give a new product a vector on day one, but behaviour usually predicts purchases better once it exists. I give the model an input such as item age or purchase count, so it can rely on content when history is thin and on behaviour as it grows. Training data must contain items at every age, which feature logging provides.