# The best embedding model for technical documentation

> API references, guides and changelogs are full of identifiers, error codes and version numbers that embedding models handle badly. How to chunk developer docs, which models to test, and a benchmark on open-source documentation.

Published 2026-10-06, updated 2026-11-14 by Truvec team. Canonical: https://truvec.dev/blog/best-embedding-model-for-technical-documentation/

Documentation assistants are one of the most common RAG applications: "ask the docs" boxes, support bots, IDE helpers. They're also a tough test for embedding models. Developer documentation mixes prose with code, and the questions people ask are full of exact tokens: a function name, an error code, a version number.

This post explains what makes technical docs different, how to prepare them, and what to test. The last section benchmarks the main models on open-source developer documentation.

## What makes technical docs different

**Identifiers carry the meaning.** `starting_after`, `E_CONN_RESET`, `--max-retries`: to a developer these are precise; to an embedding model they're rare tokens split into fragments. Two endpoints with similar names can end up closer to each other than to the question about one of them.

**Versions.** "How do I configure retries?" has a different answer in v3 and v4. Changelogs and migration guides mention old and new behavior side by side, which creates convincing wrong answers.

**Code blocks.** A chunk that's mostly code may hold the answer, but its embedding mostly reflects syntax. Models trained on code and text together handle this better.

**Short, vague questions.** Developers type "pagination", "timeout per request" or paste an error message. Short queries give the model little to work with.

**Repetition.** API references repeat the same structure for every endpoint (parameters, returns, errors), so many chunks look alike.

## Prepare the docs

- **Chunk by heading.** Split Markdown on its headings and keep the heading path in the chunk ("Client > Configuration > Timeouts"). Headings are often the best match for a question.
- **Keep code with its explanation.** Don't separate a code example from the paragraph that explains it; the paragraph gives the embedding its meaning.
- **Tag versions.** Store the product version in each chunk's metadata and filter on it, or index only the current version.
- **Add keyword search for exact tokens.** Error codes and function names are matched best by keyword search. Hybrid retrieval fixes the cases embeddings miss.

## Build questions that look like real ones

Generated questions tend to be clean and complete. Real developer questions aren't. Mix in:

- pasted error messages ("ECONNRESET when uploading large file");
- bare identifiers ("starting_after");
- "how do I" questions phrased without the API's vocabulary;
- questions about behavior that changed between versions.

If you have search logs from your docs site, they're the best source of real questions.

## What to test

Start with the large models of each provider and a couple of cheap ones. For developer docs, also test:

- **Long-input models** if your pages are long, so code-heavy chunks aren't truncated;
- **Open-weight models** if the assistant ships inside your product or runs on-premises;
- **A cheap model** as a baseline: on well-structured docs, a cheap model sometimes ties with an expensive one.

The benchmark below runs nine models on open-source developer documentation. Look at [hit rate](/blog/rag-evaluation-metrics-that-matter/) at position 1: docs assistants usually show the top result first, so a model that ranks the answer second loses more than its MRR suggests.

## Benchmark: Embedding models on developer documentation

Real benchmark generated by Truvec (benchmark #5, run 2026-11-14). Raw data: [benchmark.json](https://truvec.dev/blog/best-embedding-model-for-technical-documentation/benchmark.json), [benchmark.csv](https://truvec.dev/blog/best-embedding-model-for-technical-documentation/benchmark.csv). Full report with every question: [report.pdf](https://truvec.dev/blog/best-embedding-model-for-technical-documentation/report.pdf).

- Corpus: Docker documentation: manuals, FAQs, billing and support pages (https://github.com/docker/docs), 261 documents, 1,944 chunks
- Chunking: recursive, 1000 chars, 200 overlap
- Golden set: 100 questions generated by gpt-5.6-luna, human-reviewed
- Retrieval: top 5 by cosine similarity; models ranked by MRR, then recall@k

### Key results

- **Voyage 4 Large** ranks first, with an MRR of 0.922 and a hit rate of 99.0% in the top 5.
- One model is within 0.02 MRR of the winner (Voyage 4): on this corpus they are practically tied.
- Hit rate ranges from 90.0% to 100.0% across the 9 models.
- Cheapest model within 2 points of the winner's hit rate: **Voyage 4**, $0.02 to embed this corpus against $0.04.
- Fastest to embed a query: **OpenAI text-embedding-3-small** (4.4 ms per query, measured in batch from our servers).

### Retrieval quality

| Rank | Model | Provider | Hit rate@5 | Recall@5 | MRR | nDCG@5 | Precision@5 |
| --- | --- | --- | ---: | ---: | ---: | ---: | ---: |
| 1 | Voyage 4 Large | voyage | 99.0% | 97.5% | 0.922 | 0.924 | 20.6% |
| 2 | Voyage 4 | voyage | 100.0% | 99.5% | 0.915 | 0.929 | 21.2% |
| 3 | Gemini Embedding 2 | google | 99.0% | 98.5% | 0.876 | 0.897 | 21.0% |
| 4 | OpenAI text-embedding-3-large | openai | 99.0% | 97.5% | 0.867 | 0.885 | 20.6% |
| 5 | Qwen3 Embedding 8B | fireworks | 99.0% | 98.0% | 0.860 | 0.884 | 20.8% |
| 6 | Cohere Embed v4 | cohere | 98.0% | 96.5% | 0.859 | 0.875 | 20.4% |
| 7 | BGE Base EN v1.5 | fireworks | 95.0% | 92.5% | 0.773 | 0.802 | 19.4% |
| 8 | OpenAI text-embedding-3-small | openai | 95.0% | 93.5% | 0.769 | 0.807 | 19.8% |
| 9 | Nomic Embed Text v1.5 | fireworks | 90.0% | 89.0% | 0.765 | 0.789 | 18.8% |

### Hit rate by position

| Model | Top 1 | Top 2 | Top 3 | Top 4 | Top 5 |
| --- | ---: | ---: | ---: | ---: | ---: |
| Voyage 4 Large | 87.0% | 94.0% | 99.0% | 99.0% | 99.0% |
| Voyage 4 | 85.0% | 95.0% | 99.0% | 99.0% | 100.0% |
| Gemini Embedding 2 | 80.0% | 90.0% | 95.0% | 98.0% | 99.0% |
| OpenAI text-embedding-3-large | 79.0% | 88.0% | 94.0% | 98.0% | 99.0% |
| Qwen3 Embedding 8B | 77.0% | 89.0% | 95.0% | 99.0% | 99.0% |
| Cohere Embed v4 | 76.0% | 93.0% | 95.0% | 97.0% | 98.0% |
| BGE Base EN v1.5 | 64.0% | 85.0% | 90.0% | 92.0% | 95.0% |
| OpenAI text-embedding-3-small | 63.0% | 84.0% | 92.0% | 94.0% | 95.0% |
| Nomic Embed Text v1.5 | 67.0% | 80.0% | 86.0% | 89.0% | 90.0% |

### Cost, speed and size

| Model | Price / 1M tokens | Embed the corpus | 1,000 queries | Query embedding | Search p95 | Vectors |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Voyage 4 Large | $0.12 | $0.04 | $0.0024 | 16 ms | 18 ms | 7.6 MB |
| Voyage 4 | $0.06 | $0.02 | $0.0012 | 16 ms | 22 ms | 7.6 MB |
| Gemini Embedding 2 | $0.20 | $0.07 | $0.0040 | 18 ms | 22 ms | 22.8 MB |
| OpenAI text-embedding-3-large | $0.13 | $0.04 | $0.0026 | 9.0 ms | 61 ms | 22.8 MB |
| Qwen3 Embedding 8B | $0.10 | $0.03 | $0.0020 | 17 ms | 81 ms | 30.4 MB |
| Cohere Embed v4 | $0.12 | $0.04 | $0.0024 | 17 ms | 38 ms | 11.4 MB |
| BGE Base EN v1.5 | $0.0080 | $0.0027 | $0.0002 | 14 ms | 13 ms | 5.7 MB |
| OpenAI text-embedding-3-small | $0.02 | $0.0067 | $0.0004 | 4.4 ms | 19 ms | 11.4 MB |
| Nomic Embed Text v1.5 | $0.0080 | $0.0027 | $0.0002 | 15 ms | 16 ms | 5.7 MB |

### Score separation

| Model | Relevant chunk | Best distractor | Gap |
| --- | ---: | ---: | ---: |
| Voyage 4 Large | 0.653 | 0.607 | 0.046 |
| Voyage 4 | 0.692 | 0.646 | 0.046 |
| Gemini Embedding 2 | 0.816 | 0.792 | 0.025 |
| OpenAI text-embedding-3-large | 0.687 | 0.650 | 0.037 |
| Qwen3 Embedding 8B | 0.827 | 0.785 | 0.042 |
| Cohere Embed v4 | 0.684 | 0.638 | 0.046 |
| BGE Base EN v1.5 | 0.806 | 0.782 | 0.023 |
| OpenAI text-embedding-3-small | 0.715 | 0.687 | 0.027 |
| Nomic Embed Text v1.5 | 0.842 | 0.815 | 0.027 |

### Example questions

Rank of the first relevant chunk per model ("miss": not in the top 5).

| Question | Voyage 4 Large | Voyage 4 | Gemini Embedding 2 | OpenAI text-embedding-3-large | Qwen3 Embedding 8B | Cohere Embed v4 | BGE Base EN v1.5 | OpenAI text-embedding-3-small | Nomic Embed Text v1.5 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| How do I push an image to Docker Hub, and what tagging restriction applies? | miss | 5 | 5 | miss | 4 | miss | 3 | 1 | 4 |
| How can I use a value from a `.env` file inside a Docker Compose service's environment settings? | 1 | 1 | 3 | 1 | 1 | 2 | 5 | 2 | miss |
| What file and service configuration are required for Docker Hub automated tests, and how is the test result determined? | 1 | 1 | 4 | 2 | 4 | miss | 5 | miss | 2 |
| Which organizational roles and subscription plans are needed to create organization access tokens in Docker? | 1 | 1 | 1 | 4 | 2 | 1 | miss | miss | miss |
| When both JIT and SCIM are enabled, how are organization roles and account deactivation handled? | 1 | 1 | 1 | 5 | miss | 3 | 3 | miss | 3 |
| Which Docker Desktop versions are supported under the Pro, Team, and Business plans? | 1 | 1 | 1 | 2 | 1 | 1 | 1 | 3 | 4 |
| How do I create, publish, and share a custom MCP catalog with my team? | 1 | 1 | 1 | 1 | 3 | 5 | 2 | 2 | 3 |
| Where can I inspect all Docker Desktop settings that have been applied, including settings not visible in the GUI? | 1 | 1 | 1 | 1 | 3 | 1 | 4 | 2 | 1 |

### Notes

- The corpus is a subset of the public Docker documentation (Apache 2.0), taken from github.com/docker/docs at commit 553482c: the Desktop, Docker Hub, Compose, accounts, security and subscription and billing manuals, the FAQs, the support pages and the getting-started guides. Release notes and pages shorter than 600 characters were left out, and pages were cut at 40,000 characters.
- Several posts on this blog use this corpus and the same 100 questions, each with its own selection of models. A model can differ by a question between two of these runs: questions are embedded again at each run, and vector search is approximate.
- With 100 questions, a gap of a couple of points between two models is within the noise.

### Metric definitions

- **Hit rate**: Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
- **Recall**: Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
- **MRR**: Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
- **nDCG**: Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
- **Precision**: Share of the top k results that are relevant. Low precision means more noise sent to the LLM.
