Representation
Dense vectors compress useful features into a form that can be compared, clustered, classified, or indexed.
Engineer Guide · Updated 2026
Embeddings are a retrieval system, not just a vector.
The model matters, but production quality is shaped by the entire chain: corpus preparation, chunking, query instructions, vector dimensionality, distance metric, index configuration, metadata filters, reranking, and evaluation.
Production retrieval pipeline
content → chunk → embed → index
query → embed → retrieve → filter → rerank
measure → relevance + latency + cost + answer quality
query → embed → retrieve → filter → rerank
measure → relevance + latency + cost + answer quality
ModelChunkingDimensionIndextop-kReranker
Foundations
What an embedding actually gives you
An embedding maps an object: text, image, code, entity, or other content: to a numeric vector. The useful property is not the vector itself; it is a learned geometry in which the model can place related items near each other for a specific objective.
Similarity
Cosine similarity, dot product, or distance turns semantic relatedness into a ranking operation. The correct metric depends on how the model was trained.
Retrieval
At scale, approximate-nearest-neighbor indexes trade exactness for latency and memory. That tradeoff becomes part of the application design.
One important correction
Embeddings do not “solve the curse of dimensionality.” They create a learned lower-dimensional representation that is more useful than sparse symbolic encodings for many tasks. High-dimensional vector search still has its own scaling and indexing challenges.
Model landscape
Choose a representation family before choosing a brand
Historical models explain the evolution; production selection should focus on task fit, deployment constraints, and measured retrieval quality.
| Family | Examples | What it represents well | Best fit | Watch-outs |
|---|---|---|---|---|
| Static word vectors | Word2Vec, GloVe, FastText | Lexical relationships | Legacy NLP, lightweight features, education | One vector per word; limited context sensitivity |
| Contextual encoders | BERT-family encoders | Token meaning in context | NLP features, classification, specialized encoders | Raw BERT pooling is not automatically a strong retrieval embedding |
| Bi-encoder sentence / passage | Sentence Transformers, E5/BGE-style families | Sentence and passage similarity | Semantic search, RAG retrieval, clustering | Instruction/prefix format and hard negatives can materially affect quality |
| Hosted text embeddings | OpenAI text-embedding-3-small / text-embedding-3-large | General text and multilingual retrieval | Managed embedding API, fast integration | External API cost, privacy boundary, model-version governance |
| Late interaction / multi-vector | ColBERT-style retrieval | Fine-grained token-level matching | Higher-precision retrieval and reranking | Larger indexes and more compute than single-vector retrieval |
| Multimodal | CLIP-family models | Cross-modal relationships | Text↔image search and multimodal retrieval | Shared-space quality varies by domain and modality |
Evaluation
Use public benchmarks to shortlist. Use your workload to decide.
MTEB now spans a large, multilingual collection of embedding tasks and is useful for screening. It is not a substitute for testing the corpus, query patterns, filters, and downstream outcomes you will operate.
Offline retrieval set
- Representative user queries
- Known relevant documents or passages
- Hard negatives and near-miss documents
- Metadata-filtered queries
- Long-tail and ambiguous queries
Metrics that matter
- Recall@k / Precision@k
- nDCG / MRR for ranking quality
- p50 / p95 / p99 latency
- Index build and update freshness
- Embedding + storage + retrieval + reranking cost
- RAG groundedness / answer usefulness
Evaluate the pipeline, not only the embedding
A “better” embedding model can lose in production if it needs a larger index, adds latency, handles metadata filters poorly, or improves retrieval metrics without improving the final answer. Run controlled variants end-to-end.
Production playbook
The knobs that actually change retrieval quality
1 · Corpus
Document cleanup, deduplication, permissions, metadata, language, and freshness.
2 · Chunking
Boundary method, chunk length, overlap, titles, hierarchy, and contextual enrichment.
3 · Model
Embedding family, instruction/prefix format, dimension, normalization, domain adaptation.
4 · Retrieval
Distance metric, vector store, index family, search breadth, metadata filters, hybrid search.
5 · Post-retrieval
top-k, score threshold, deduplication, diversity, reranking, context packing.
6 · Operations
Batching, caching, refresh cadence, backfills, monitoring, rollback, and cost budgets.
7 · Evaluation
Golden sets, online experiments, failure slices, human review, and regression thresholds.
8 · Versioning
Model IDs, corpus version, chunk recipe, index settings, and an explicit migration plan.
DataKnobs pattern
KREATE builds it. KONTROLS governs it. KNOBS improves it.
KREATE
Build ingestion, chunking, embedding, vector indexing, retrieval, reranking, and RAG workflows around enterprise data.
KONTROLS
Track source permissions, model/version lineage, evaluation evidence, privacy boundaries, approvals, and deployment state.
KNOBS
Expose model, chunking, dimensions, index parameters, top-k, thresholds, reranking, and routing as measurable levers instead of hidden constants.
Safe model upgrades
Do not overwrite an embedding model in place. Build a new index or namespace, re-embed the corpus, evaluate it against the current version, shadow or canary traffic, then cut over with rollback preserved.
FAQ
Production embedding questions
Test both, but chunking is often the cheaper first lever. Hold the corpus and query set constant, compare chunk size and boundaries, then compare model families on the same evaluation set.
No. MTEB is useful for screening model families and tasks, but production selection should use your corpus, query distribution, filters, latency target, cost constraints, and downstream answer quality.
Treat that as unsafe unless the model documentation explicitly guarantees compatible vector spaces. A model change normally requires re-embedding and an explicit cutover or dual-index migration.
Use the metric recommended for the model and index. For normalized embeddings, cosine similarity and dot product are closely related; changing metrics without validation can change ranking behavior.
At minimum: retrieval relevance such as Recall@k, nDCG or MRR; filtered-query behavior; p50/p95 latency; ingestion freshness; cost; and downstream RAG or search outcomes on a representative test set.
Continue