Turning 80,000 untagged product photos into a searchable catalog
80,000 raw product photos, transformed into a searchable, self-describing catalog.
Challenge
The client’s catalog was 80,000+ product images with barely any metadata, invisible to search, impossible to recommend from. We built a multimodal enrichment pipeline that makes the images describe themselves: OpenCLIP generates visual-similarity embeddings, BLIP-3 writes accurate captions automatically, and Segment Anything v2 isolates products from cluttered photography.
Approach
The enriched embeddings are indexed into ChromaDB and Vespa, unlocking the features shoppers expect from a modern storefront: semantic text search that understands intent, "find similar" visual discovery, and personalized recommendations, all computed offline against the full catalog, with no real-time inference constraints.
Outcome
Every product image now describes itself with captions and embeddings, making the full catalog searchable by meaning rather than keywords.
Shoppers get semantic search, "find similar" visual discovery and personalised recommendations, all computed offline with no real-time inference costs.
Key results
- 80,000+ images enriched end to end
- OpenCLIP + BLIP-3 + SAM-2 multimodal pipeline
- Vector search indexed in ChromaDB and Vespa
- Semantic search, find-similar, and recommendations
Gallery