The best embedding model for RAG: 16 models compared on one knowledge base

There is no single best embedding model for RAG, but there is a best one for your documents, budget and constraints. How the main model families differ, which trade-offs matter, and a 16-model benchmark on a real knowledge base.

  • embeddings
  • rag
  • benchmark
  • model-comparison

Download the benchmark report (PDF)

"Which embedding model is best for RAG?" has an honest answer: the one that retrieves the right passages from your documents, at a cost and latency you can live with. That sounds like a dodge, but it's a practical rule. The ranking of embedding models changes from one corpus to another, and the gap between the top models is usually smaller than the gap created by your domain, your chunking and your users' questions.

This post gives you a map of the options, the trade-offs that decide between them, and a starting shortlist per situation. The last section benchmarks all 16 models Truvec runs on the same knowledge base.

The model families

Embedding models for retrieval fall into a few groups:

Family Models in Truvec What sets them apart
OpenAI text-embedding-3-small, text-embedding-3-large, ada-002 The default in most tutorials. 3-small is very cheap; both v3 models can shorten their vectors
Voyage AI Voyage 4 Large, Voyage 4, Voyage 4 Lite Retrieval-focused models in three price tiers, with separate query and document modes
Cohere Embed v4, Embed English v3, Embed Multilingual v3 Strong multilingual line-up; v3 models read up to 512 tokens per text
Google Gemini Embedding 2, Gemini Embedding 001 Large 3,072-dimension vectors and long inputs
Open source Qwen3 Embedding 8B, BGE Base, Nomic, mxbai, GTE Large Open weights: you can self-host them. Cheapest per token through a hosted API

Older models such as text-embedding-ada-002 are still in many production systems. Including them in a comparison tells you what a migration would gain.

The trade-offs that decide

Retrieval quality on your data. Measure it with hit rate and MRR at the number of chunks you actually send to the LLM. How to choose an embedding model for RAG explains the metrics and how to build the test set.

Cost. Embedding prices differ by more than 10× between the cheapest and the most expensive models. You pay to embed the whole corpus once, again on every re-embedding (new chunking, new model), and a little per query. For most RAG systems the corpus is the bigger bill.

Vector size. Dimensions range from 768 to 4,096. Storage and memory grow linearly with them: at a million chunks, 4,096 dimensions take about 16 GB of raw float32 vectors, against 3 GB at 768. If your vector database keeps vectors in RAM, this is a monthly cost.

Input length. Some models read only 512 tokens per chunk and silently ignore the rest; others read thousands. If your chunks are long, a short-input model is evaluated on truncated text.

Languages. English-only models fall apart on other languages. If any part of your corpus or your users isn't English, test multilingual models.

Hosting. API models are simple to run but send your text to a provider. Open-weight models can run in your own infrastructure, which may be a hard requirement for regulated data.

A starting shortlist

Use this to decide what to test, not what to pick:

  • Tight budget, English content: OpenAI text-embedding-3-small, Voyage 4 Lite, and an open model such as BGE Base or Nomic.
  • Quality first: the large models from each provider (Voyage 4 Large, OpenAI 3-large, Cohere Embed v4, Gemini Embedding 2) plus Qwen3 Embedding 8B.
  • Multilingual: Cohere Embed v4 or Multilingual v3, Gemini, Qwen3, and OpenAI 3-large as a reference.
  • Must self-host: Qwen3 Embedding, BGE, Nomic, mxbai and GTE, compared with an API model to see what you give up.
  • Migrating off ada-002: ada-002 itself, 3-small and 3-large, so the comparison shows the gain on your own data.

Why the winner changes between corpora

Run the same 16 models on product docs, contracts and support tickets, and you'll get three different orderings. Models differ in how they handle rare vocabulary (product names, legal terms, error codes), long passages, and short or vague questions. The differences that look large on a leaderboard often vanish on your data, and the ones that matter for you rarely show up on a leaderboard.

Two habits keep the decision honest:

  1. Treat close results as ties. With 100 questions, a difference of a couple of points is within the noise. Pick between tied models on price, latency and storage.
  2. Re-run when things change. New models ship several times a year, and your corpus grows. A comparison that takes minutes is worth repeating.

The benchmark below runs all 16 models on one knowledge base. Use it as an example of what a comparison shows, then run it on your own documents.

Real benchmark - generated by Truvec

Benchmark: All 16 Truvec models on a product knowledge base

Corpus
Docker documentation: manuals, FAQs, billing and support pages
Documents
261
Chunks
1,944
Chunking
recursive, 1000 chars, 200 overlap
Golden set
100 questions, human-reviewed
Models
16 compared
Top k
5
Run
2026-11-14

Download the full report (PDF) Every question, with the passages each model retrieved.

Key results

  • Voyage 4 Large ranks first, with an MRR of 0.922 and a hit rate of 99.0% in the top 5.
  • One model is within 0.02 MRR of the winner (Voyage 4): on this corpus they are practically tied.
  • Hit rate ranges from 90.0% to 100.0% across the 16 models.
  • Cheapest model within 2 points of the winner's hit rate: Voyage 4 Lite, $0.0067 to embed this corpus against $0.04.
  • Fastest to embed a query: OpenAI text-embedding-ada-002 (5.5 ms per query, measured in batch from our servers).

Retrieval quality

Retrieval quality at top 5, ranked by MRR, then recall@k
ModelHit rate@5Recall@5MRRnDCG@5Precision@5
1Voyage 4 Largevoyage - 1024 dims99.0%97.5%0.9220.92420.6%
2Voyage 4voyage - 1024 dims100.0%99.5%0.9150.92921.2%
3Voyage 4 Litevoyage - 1024 dims99.0%97.5%0.8930.90720.6%
4Gemini Embedding 2google - 3072 dims99.0%98.5%0.8760.89721.0%
5Gemini Embedding 001google - 3072 dims98.0%97.5%0.8690.89020.8%
6OpenAI text-embedding-3-largeopenai - 3072 dims99.0%97.5%0.8670.88520.6%
7Cohere Embed v4cohere - 1536 dims98.0%96.5%0.8590.87520.4%
8Qwen3 Embedding 8Bfireworks - 4096 dims99.0%98.0%0.8580.88320.8%
9Cohere Embed Multilingual v3cohere - 1024 dims94.0%92.5%0.8480.86119.6%
10mxbai Embed Large v1fireworks - 1024 dims96.0%93.5%0.8170.83719.6%
11Cohere Embed English v3cohere - 1024 dims97.0%95.0%0.8130.83820.0%
12GTE Largefireworks - 1024 dims93.0%90.5%0.7920.81019.0%
13BGE Base EN v1.5fireworks - 768 dims95.0%92.5%0.7730.80219.4%
14OpenAI text-embedding-3-smallopenai - 1536 dims95.0%93.5%0.7690.80719.8%
15Nomic Embed Text v1.5fireworks - 768 dims90.0%89.0%0.7650.78918.8%
16OpenAI text-embedding-ada-002openai - 1536 dims93.0%91.5%0.7130.75819.4%

Hit rate by position

How many questions the first results already answer. A model that finds the answer at position 1 lets you send fewer chunks to the LLM.

Hit rate by position: share of questions answered by the first j results
ModelTop 1Top 2Top 3Top 4Top 5
1Voyage 4 Largevoyage - 1024 dims87.0%94.0%99.0%99.0%99.0%
2Voyage 4voyage - 1024 dims85.0%95.0%99.0%99.0%100.0%
3Voyage 4 Litevoyage - 1024 dims81.0%95.0%99.0%99.0%99.0%
4Gemini Embedding 2google - 3072 dims80.0%90.0%95.0%98.0%99.0%
5Gemini Embedding 001google - 3072 dims78.0%92.0%97.0%98.0%98.0%
6OpenAI text-embedding-3-largeopenai - 3072 dims79.0%88.0%94.0%98.0%99.0%
7Cohere Embed v4cohere - 1536 dims76.0%93.0%95.0%97.0%98.0%
8Qwen3 Embedding 8Bfireworks - 4096 dims77.0%88.0%95.0%99.0%99.0%
9Cohere Embed Multilingual v3cohere - 1024 dims78.0%88.0%92.0%94.0%94.0%
10mxbai Embed Large v1fireworks - 1024 dims73.0%84.0%88.0%94.0%96.0%
11Cohere Embed English v3cohere - 1024 dims71.0%84.0%92.0%94.0%97.0%
12GTE Largefireworks - 1024 dims68.0%86.0%91.0%93.0%93.0%
13BGE Base EN v1.5fireworks - 768 dims64.0%85.0%90.0%92.0%95.0%
14OpenAI text-embedding-3-smallopenai - 1536 dims63.0%84.0%92.0%94.0%95.0%
15Nomic Embed Text v1.5fireworks - 768 dims67.0%80.0%86.0%89.0%90.0%
16OpenAI text-embedding-ada-002openai - 1536 dims55.0%79.0%88.0%93.0%93.0%

Cost, speed and size

Embedding prices are the providers' list prices. Corpus cost is a one-time cost per model; the vector size is raw float32 storage, before the vector database's own overhead.

Cost, speed and size on this corpus
ModelPrice / 1M tokensEmbed the corpus1,000 queriesQuery embeddingSearch p95Vectors
1Voyage 4 Largevoyage - 1024 dims$0.12$0.04$0.002417 ms16 ms7.6 MB
2Voyage 4voyage - 1024 dims$0.06$0.02$0.001211 ms24 ms7.6 MB
3Voyage 4 Litevoyage - 1024 dims$0.02$0.0067$0.000417 ms20 ms7.6 MB
4Gemini Embedding 2google - 3072 dims$0.20$0.07$0.004020 ms27 ms22.8 MB
5Gemini Embedding 001google - 3072 dims$0.15$0.05$0.003016 ms23 ms22.8 MB
6OpenAI text-embedding-3-largeopenai - 3072 dims$0.13$0.04$0.00268.0 ms30 ms22.8 MB
7Cohere Embed v4cohere - 1536 dims$0.12$0.04$0.002414 ms54 ms11.4 MB
8Qwen3 Embedding 8Bfireworks - 4096 dims$0.10$0.03$0.002010 ms33 ms30.4 MB
9Cohere Embed Multilingual v3cohere - 1024 dims$0.10$0.03$0.002011 ms17 ms7.6 MB
10mxbai Embed Large v1fireworks - 1024 dims$0.02$0.0054$0.000318 ms16 ms7.6 MB
11Cohere Embed English v3cohere - 1024 dims$0.10$0.03$0.00208.9 ms16 ms7.6 MB
12GTE Largefireworks - 1024 dims$0.02$0.0054$0.000318 ms24 ms7.6 MB
13BGE Base EN v1.5fireworks - 768 dims$0.0080$0.0027$0.000214 ms15 ms5.7 MB
14OpenAI text-embedding-3-smallopenai - 1536 dims$0.02$0.0067$0.00049.4 ms41 ms11.4 MB
15Nomic Embed Text v1.5fireworks - 768 dims$0.0080$0.0027$0.000213 ms19 ms5.7 MB
16OpenAI text-embedding-ada-002openai - 1536 dims$0.10$0.03$0.00205.5 ms27 ms11.4 MB

Score separation

Average similarity of the best relevant chunk and of the best irrelevant one. A wide gap makes a similarity threshold reliable; a narrow one means right and wrong chunks look alike to the model.

Score separation: how clearly each model scores the right chunk above the best wrong one
ModelRelevant chunkBest distractorGap
1Voyage 4 Largevoyage - 1024 dims0.6530.6070.046
2Voyage 4voyage - 1024 dims0.6920.6460.046
3Voyage 4 Litevoyage - 1024 dims0.6650.6220.044
4Gemini Embedding 2google - 3072 dims0.8160.7920.025
5Gemini Embedding 001google - 3072 dims0.7960.7810.015
6OpenAI text-embedding-3-largeopenai - 3072 dims0.6870.6500.037
7Cohere Embed v4cohere - 1536 dims0.6840.6380.046
8Qwen3 Embedding 8Bfireworks - 4096 dims0.8270.7850.042
9Cohere Embed Multilingual v3cohere - 1024 dims0.7880.7470.041
10mxbai Embed Large v1fireworks - 1024 dims0.8420.8170.026
11Cohere Embed English v3cohere - 1024 dims0.7770.7410.037
12GTE Largefireworks - 1024 dims0.9290.9170.012
13BGE Base EN v1.5fireworks - 768 dims0.8060.7820.023
14OpenAI text-embedding-3-smallopenai - 1536 dims0.7150.6870.027
15Nomic Embed Text v1.5fireworks - 768 dims0.8420.8150.027
16OpenAI text-embedding-ada-002openai - 1536 dims0.8710.8610.009

Example questions

Rank of the first relevant chunk for each model; miss means it wasn't in the top 5. These are the questions where the models disagree most.

Example questions from the golden set
QuestionVoyage 4 LargeVoyage 4Voyage 4 LiteGemini Embedding 2Gemini Embedding 001OpenAI text-embedding-3-largeCohere Embed v4Qwen3 Embedding 8BCohere Embed Multilingual v3mxbai Embed Large v1Cohere Embed English v3GTE LargeBGE Base EN v1.5OpenAI text-embedding-3-smallNomic Embed Text v1.5OpenAI text-embedding-ada-002
How do I push an image to Docker Hub, and what tagging restriction applies?
Expected passage
## Push an image to Docker Hub Select the image from the list, select the **More options** button and select **Push to Hub**. > [!NOTE] > > You can only push an image to Docker Hub if the image belongs to your Docker ID or your organization. That is, the image must contain the co
miss5miss5missmissmiss44424314miss
When both JIT and SCIM are enabled, how are organization roles and account deactivation handled?
Expected passage
Roles don't move back and forth. JIT sets the organization role only when it first adds the user. A later SCIM update can change that role, and the next sign-in leaves the SCIM role in place. When a user isn't assigned to the Docker application in the IdP, the next synchronizatio
1121153miss35243miss34
How do I create, publish, and share a custom MCP catalog with my team?
Expected passage
Create a catalog and specify which servers to include: ```console $ docker mcp catalog create registry.example.com/mcp/data-tools:latest \ --title "Data Analysis Tools" \ --server catalog://mcp/docker-mcp-catalog/sequentialthinking \ --server catalog://mcp/docker-mcp-catalog/brav
11111153miss4522233
How do I search Docker Desktop logs, move between matching entries, and clear the terminal?
Expected passage
### Logs Select **Logs** to view output from the container in real time. While viewing logs, you can: - Use `Cmd + f`/`Ctrl + f` to open the search bar and find specific entries. Search matches are highlighted in yellow. - Press `Enter` or `Shift + Enter` to jump to the next or p
11123423miss1521121
When multiple Compose files are supplied with `-f`, how are their service settings combined, and which file takes precedence?
Expected passage
Compose merges files in the order they're specified on the command line. Subsequent files may merge, override, or add to their predecessors. For example: ```console $ docker compose -f compose.yaml -f compose.admin.yaml run backup_db ``` The `compose.yaml` file might specify a `w
12311142miss53missmiss4missmiss
Which organizational roles and subscription plans are needed to create organization access tokens in Docker?
Expected passage
> [!WARNING] > > OATs don't work with Docker Desktop or Image Access Management. For > those features, use > PATs instead. ## Best practices - Set an expiration date and rotate tokens on a schedule - Grant only the repositories and scopes each job needs - Review the **Last used**
111114123missmissmissmissmissmissmiss
How do I make a container service accessible from my host computer using Docker port publishing?
Expected passage
# Publishing and exposing ports ## Explanation If you've been following the guides so far, you understand that containers provide isolated processes for each component of your application. Each component - a React frontend, a Python API, and a Postgres database - runs in its own
3334miss113missmiss1213miss2
How can I confirm that an image migration preserved the same digest, including for multi-platform images?
Expected passage
```bash #!/usr/bin/env bash set -euo pipefail SRC_ORG="oldorg" DEST_ORG="neworg" REPO="myapp" TAG="1.2.3" SRC_IMAGE="${SRC_ORG}/${REPO}:${TAG}" DEST_IMAGE="${DEST_ORG}/${REPO}:${TAG}" # Using crane SRC_DIGEST=$(crane digest "${SRC_IMAGE}") DEST_DIGEST=$(crane digest "${DEST_IMAGE
11111111235miss31miss1

Notes

  • The corpus is a subset of the public Docker documentation (Apache 2.0), taken from github.com/docker/docs at commit 553482c: the Desktop, Docker Hub, Compose, accounts, security and subscription and billing manuals, the FAQs, the support pages and the getting-started guides. Release notes and pages shorter than 600 characters were left out, and pages were cut at 40,000 characters.
  • Several posts on this blog use this corpus and the same 100 questions, each with its own selection of models. A model can differ by a question between two of these runs: questions are embedded again at each run, and vector search is approximate.
  • With 100 questions, a gap of a couple of points between two models is within the noise.
  • BGE Base, mxbai Embed Large and GTE Large read at most 380 tokens per chunk in Truvec, and the two Cohere v3 models 512. Chunks average about 170 tokens, so little or nothing was truncated.

Methodology

Truvec split the documents into chunks, generated a golden set of questions with gpt-5.6-luna (each question is linked to the chunks that answer it), embedded the corpus and the questions with every model, and retrieved the top 5 chunks for each question by cosine similarity. Every model saw the same chunks and the same questions. Scores are averaged over all 100 questions.

Hit rate
Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
Recall
Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
MRR
Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
nDCG
Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
Precision
Share of the top k results that are relevant. Low precision means more noise sent to the LLM.

Downloads: full report (PDF) - benchmark.json - benchmark.csv - this post as Markdown

Voyage 4 Large won here, on these documents. Find the model that wins on yours: run the same benchmark on your documents with Truvec.