Visual / Multimodal Search

01What comes in — a photo, text, or both?

A photo in, ranked visually similar products out — and text or photo-plus-text queries hit the same index, because the image and text encoders share one embedding space.

02The photo has a couch AND a lamp — what do we search for?

An object detector finds each shoppable item first; the user taps one and we embed that crop — embedding the whole scene would blend couch, lamp, and rug into one muddy vector.

03Similar in style, or this exact product?

Both modes: visual similarity is the default, and an exact-match tier (a tight near-duplicate distance threshold) answers “buy this exact chair” — near-dups are also collapsed so the page is not the same image twenty times.

04A merchant lists a product at noon — searchable when?

Within minutes: new images stream through detect-and-embed into a fresh index segment, and retrieval needs zero clicks or history, because the content embedding IS the cold-start signal.

05Do results improve as shoppers click?

Yes: clicks and add-to-carts on result positions become training pairs for the re-ranker and, in bulk, fine-tuning data for the next encoder version — behavior sharpens what pixels started.

06One embedding per feature, or one for everything?

One unified multi-task embedding powers crop search, camera search, and exact match — improvements propagate to every surface and we run one migration, not five, at the cost of some per-task peak accuracy.

01How fast must a camera search feel?

p95 around 250 ms end-to-end, with a per-stage budget — detect, encode, retrieve, and re-rank each own a slice, so “search is slow” always has a named culprit.

02How much recall may compression cost us?

ANN recall@100 stays at 95% or better against exact search, measured continuously — quantization saves RAM only if re-ranking the shortlist at full precision buys the accuracy back.

03Billions of vectors — what is the memory story?

The index fits a memory budget, not “whatever it takes”: product quantization shrinks each vector roughly 30×, so the catalog lives on a handful of machines instead of a fleet.

04Streaming adds all day — does the index rot?

Continuous upserts land in a small fresh segment while the big immutable index rebuilds on schedule — graph indexes degrade under endless inserts, and rebuild quality is measurable.

05A new encoder ships — can old and new vectors mix?

Vectors are only comparable within one model version, so every index is version-stamped and encoder upgrades run dual-index with shadow evaluation before cutover.