The 30 second answer
More dimensions capture more nuance but cost more to train, store and search, so I size by whether serving cost or quality dominates. Matryoshka training lets one vector's prefixes work alone, so a short prefix retrieves and the full vector reranks. Retraining creates an incompatible space, so I re-embed the corpus into a new index and switch atomically. Freshness needs batch re-embedding plus event-driven updates.
Size. Larger vectors cost memory, index size and latency on every query. High-traffic recommendation retrieval leans small; tasks where quality matters more than cost lean large. Opinion: choose by measuring recall and latency at a few sizes rather than by a rule of thumb.
Matryoshka embeddings. Matryoshka Representation Learning trains one vector so that nested prefixes are each usable embeddings. Training computes the loss on several prefix lengths and sums them, optionally weighted, which pushes important information to the front. Use a short prefix for coarse retrieval and the full vector to rerank the short list. Sentence Transformers exposes this as MatryoshkaLoss wrapped around another loss.
Versions. Vectors from two model versions live in different spaces; dimension 7 of one is not dimension 7 of the other. Querying an old index with new query vectors still returns results, they are just meaningless, and the only symptom may be a drop in aggregate metrics. Version every vector and index, re-embed the corpus with the new model into a new index, validate offline, then switch query and index together, keeping the old pair for rollback. A provider retiring a hosted model forces the same full re-embed.
Staleness. Titles change and tastes drift. Run a scheduled batch re-embed plus an event-driven path for important updates. Where behaviour shifts within minutes, such as short video, news or fraud, nightly is too slow. ByteDance's Monolith is built for this: it trains online from streaming user feedback instead of separating batch training from serving, and uses a collisionless embedding table with expirable embeddings and frequency filtering to bound memory.