# The best embedding model for legal documents: contracts, clauses and regulations

> Legal text is hard for embedding models: long clauses, defined terms, cross-references and near-identical boilerplate. What makes legal retrieval different, how to chunk and test it, and a benchmark on contracts and regulations.

Published 2026-10-06, updated 2026-11-14 by Truvec team. Canonical: https://truvec.dev/blog/best-embedding-model-for-legal-documents/

Legal RAG has to be precise. A lawyer asking whether a contract can be terminated without cause needs *the* termination clause of *that* contract, not a similar clause from another agreement and not the definition of "Term". Embedding models built and benchmarked on web text struggle with exactly what makes legal documents legal.

This post covers what makes legal retrieval hard, how to prepare legal documents for embedding, and how to test models on them. The last section is a benchmark on public contracts and regulations.

## Why legal text is hard for embeddings

**Boilerplate everywhere.** Contracts reuse the same clauses with small changes: a liability cap of 12 months of fees in one agreement, of the total contract value in another. Embeddings place near-identical clauses next to each other, so the right clause competes with many convincing look-alikes. These are the hardest distractors there are.

**Meaning hides in small words.** "Shall", "may", "unless", "except as provided in Section 9" change the meaning of a clause completely, but barely change its embedding. A model can retrieve the right topic and still return the clause that says the opposite.

**Defined terms.** "Confidential Information", "Services" and "Affiliate" are defined once, then used everywhere with their capital letter and their precise meaning. Questions use everyday words ("secret data", "the work", "sister company"), so the model has to connect the two.

**Cross-references.** "Subject to Section 12.3" sends the reader elsewhere. A chunk containing that reference is incomplete on its own, and a question about Section 12.3 by number needs exact matching, which embeddings are bad at.

**Length.** Clauses run long, and regulations nest articles, paragraphs and points. Short-input models (512 tokens) may truncate a clause before the part that matters.

## Prepare the documents

- **Chunk by structure, not by size.** Split on articles, sections and clauses. A chunk should be a complete clause with its heading ("12. Limitation of Liability"), so the heading's words help retrieval.
- **Use larger chunks than for prose.** Legal clauses need their full text to be understood. Test chunk sizes around 1,500 to 2,500 characters, and check which models truncate them.
- **Keep identity in metadata.** Store the document, party, date and jurisdiction with each chunk, and filter on them. No embedding model reliably tells two versions of the same NDA apart.
- **Add keyword search for references.** Questions like "what does Section 14.2 say?" or "Article 33 GDPR" are best answered by exact matching. Hybrid search (keywords plus embeddings) covers both.

## Test the models on what matters

A legal [golden set](/blog/prototype-rag-retrieval-strategy/) should include:

- questions phrased in plain language about defined terms;
- questions whose answer is an exception ("can the supplier subcontract?" when the rule is "no, except with written consent");
- questions that only one of several similar contracts answers;
- a few questions where the right answer is "this contract doesn't say".

Look at precision as well as [hit rate](/blog/rag-evaluation-metrics-that-matter/): in legal work, a wrong clause in the context can be worse than a missing one, because the LLM may cite it with confidence.

## Confidentiality and hosting

Legal documents are often confidential, and sending them to an embedding API may be ruled out by a client agreement or a regulator. Open-weight models (Qwen3 Embedding, BGE, GTE, mxbai, Nomic) can run in your own infrastructure. Include at least one of them in your comparison: you'll see exactly what self-hosting costs you in retrieval quality, if anything.

## What to take from the benchmark

The benchmark below uses public contracts, terms of service and regulations. Your documents will differ: a firm's M&A contracts are not consumer terms of service. Look at the gaps between models, the score separation (how clearly each model sets the right clause apart from look-alikes), and the example questions. Then run the same comparison on a sample of your own documents before you choose.

## Benchmark: Embedding models on contracts and regulations

Real benchmark generated by Truvec (benchmark #14, run 2026-11-14). Raw data: [benchmark.json](https://truvec.dev/blog/best-embedding-model-for-legal-documents/benchmark.json), [benchmark.csv](https://truvec.dev/blog/best-embedding-model-for-legal-documents/benchmark.csv). Full report with every question: [report.pdf](https://truvec.dev/blog/best-embedding-model-for-legal-documents/report.pdf).

- Corpus: Commercial contracts (CUAD), GitHub terms and policies, and three EU regulations (https://www.atticusprojectai.org/cuad), 58 documents, 5,024 chunks
- Chunking: recursive, 1000 chars, 200 overlap
- Golden set: 100 questions generated by gpt-5.6-luna, human-reviewed
- Retrieval: top 5 by cosine similarity; models ranked by MRR, then recall@k

### Key results

- **Voyage 4 Large** ranks first, with an MRR of 0.828 and a hit rate of 94.0% in the top 5.
- One model is within 0.02 MRR of the winner (Voyage 4): on this corpus they are practically tied.
- Hit rate ranges from 77.0% to 94.0% across the 8 models.
- Cheapest model within 2 points of the winner's hit rate: **Voyage 4**, $0.05 to embed this corpus against $0.09.
- Fastest to embed a query: **OpenAI text-embedding-3-small** (6.0 ms per query, measured in batch from our servers).

### Retrieval quality

| Rank | Model | Provider | Hit rate@5 | Recall@5 | MRR | nDCG@5 | Precision@5 |
| --- | --- | --- | ---: | ---: | ---: | ---: | ---: |
| 1 | Voyage 4 Large | voyage | 94.0% | 94.0% | 0.828 | 0.856 | 19.2% |
| 2 | Voyage 4 | voyage | 92.0% | 91.5% | 0.815 | 0.838 | 18.6% |
| 3 | Gemini Embedding 2 | google | 88.0% | 87.5% | 0.742 | 0.773 | 17.8% |
| 4 | Cohere Embed v4 | cohere | 84.0% | 83.5% | 0.734 | 0.757 | 17.0% |
| 5 | Qwen3 Embedding 8B | fireworks | 86.0% | 86.0% | 0.705 | 0.744 | 17.6% |
| 6 | OpenAI text-embedding-3-large | openai | 83.0% | 83.0% | 0.697 | 0.729 | 17.0% |
| 7 | OpenAI text-embedding-3-small | openai | 77.0% | 76.5% | 0.656 | 0.680 | 15.6% |
| 8 | BGE Base EN v1.5 | fireworks | 78.0% | 77.5% | 0.619 | 0.654 | 15.8% |

### Hit rate by position

| Model | Top 1 | Top 2 | Top 3 | Top 4 | Top 5 |
| --- | ---: | ---: | ---: | ---: | ---: |
| Voyage 4 Large | 75.0% | 87.0% | 89.0% | 92.0% | 94.0% |
| Voyage 4 | 73.0% | 88.0% | 89.0% | 90.0% | 92.0% |
| Gemini Embedding 2 | 65.0% | 77.0% | 84.0% | 85.0% | 88.0% |
| Cohere Embed v4 | 67.0% | 75.0% | 79.0% | 81.0% | 84.0% |
| Qwen3 Embedding 8B | 61.0% | 71.0% | 81.0% | 85.0% | 86.0% |
| OpenAI text-embedding-3-large | 60.0% | 75.0% | 78.0% | 82.0% | 83.0% |
| OpenAI text-embedding-3-small | 57.0% | 70.0% | 74.0% | 77.0% | 77.0% |
| BGE Base EN v1.5 | 52.0% | 65.0% | 68.0% | 75.0% | 78.0% |

### Cost, speed and size

| Model | Price / 1M tokens | Embed the corpus | 1,000 queries | Query embedding | Search p95 | Vectors |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Voyage 4 Large | $0.12 | $0.09 | $0.0026 | 13 ms | 52 ms | 19.6 MB |
| Voyage 4 | $0.06 | $0.05 | $0.0013 | 19 ms | 23 ms | 19.6 MB |
| Gemini Embedding 2 | $0.20 | $0.15 | $0.0043 | 19 ms | 44 ms | 58.9 MB |
| Cohere Embed v4 | $0.12 | $0.09 | $0.0026 | 13 ms | 27 ms | 29.4 MB |
| Qwen3 Embedding 8B | $0.10 | $0.08 | $0.0021 | 166 ms | 72 ms | 78.5 MB |
| OpenAI text-embedding-3-large | $0.13 | $0.10 | $0.0028 | 6.6 ms | 20 ms | 58.9 MB |
| OpenAI text-embedding-3-small | $0.02 | $0.02 | $0.0004 | 6.0 ms | 32 ms | 29.4 MB |
| BGE Base EN v1.5 | $0.0080 | $0.0062 | $0.0002 | 14 ms | 23 ms | 14.7 MB |

### Score separation

| Model | Relevant chunk | Best distractor | Gap |
| --- | ---: | ---: | ---: |
| Voyage 4 Large | 0.576 | 0.506 | 0.068 |
| Voyage 4 | 0.628 | 0.561 | 0.066 |
| Gemini Embedding 2 | 0.807 | 0.783 | 0.024 |
| Cohere Embed v4 | 0.668 | 0.621 | 0.047 |
| Qwen3 Embedding 8B | 0.781 | 0.746 | 0.033 |
| OpenAI text-embedding-3-large | 0.680 | 0.634 | 0.049 |
| OpenAI text-embedding-3-small | 0.712 | 0.671 | 0.040 |
| BGE Base EN v1.5 | 0.761 | 0.744 | 0.020 |

### Example questions

Rank of the first relevant chunk per model ("miss": not in the top 5).

| Question | Voyage 4 Large | Voyage 4 | Gemini Embedding 2 | Cohere Embed v4 | Qwen3 Embedding 8B | OpenAI text-embedding-3-large | OpenAI text-embedding-3-small | BGE Base EN v1.5 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Under the Whitesmoke promotion and distribution agreement, what restrictions apply to the Distributor's treatment of products already installed by end users after the agreement ends? | 1 | 1 | 2 | 3 | 2 | miss | 4 | 4 |
| What responsibilities apply to anyone collecting personal information through the GitHub Service? | 2 | 2 | 3 | 1 | 2 | miss | miss | 4 |
| Under the video-on-demand content license between Rogers and EuroMedia, when may a party disclose the other's confidential information to comply with legal requirements, and what must it do beforehand? | 5 | 2 | 4 | 5 | 3 | miss | 4 | 4 |
| Under GitHub's Corporate Terms of Service, when can GitHub assign the agreement without the customer's consent? | 1 | 1 | 1 | 1 | 3 | 2 | miss | 4 |
| What kinds of conduct are prohibited at the GitHub event? | 1 | 1 | 2 | 2 | 3 | 1 | 1 | 5 |
| What extra charges does a sponsor pay under the program, including when using a credit card? | 1 | 1 | 1 | 1 | miss | 4 | miss | 3 |
| What uses of GitHub-hosted Actions are not allowed? | 1 | 1 | 1 | 2 | 3 | 2 | 2 | miss |
| Do I need GitHub's permission to use its trademarks or logos? | 1 | 1 | 3 | 1 | 4 | 2 | 1 | 1 |

### Notes

- The corpus has three parts. 36 commercial contracts from the CUAD dataset (The Atticus Project, CC BY 4.0, text extracted from SEC EDGAR filings), picked across contract types among those between 25,000 and 90,000 characters. 19 terms, agreements and policies from github.com/github/site-policy (CC0). The English texts of Regulation (EU) 2016/679 (GDPR), Regulation (EU) 2022/2065 (Digital Services Act) and Regulation (EU) 2024/1689 (AI Act), from the Publications Office of the EU.
- The contract texts are used as published in CUAD: page numbers, redaction marks and layout left over from the filings are still in the chunks.
- Reviewing the questions, we rewrote 27 that did not say which document they were about ("Which state’s law governs the agreement?") to name the contract or its parties, the way a lawyer would ask. The name is often absent from the chunk that holds the clause, which is part of what makes legal retrieval hard.
- With 100 questions, a gap of a couple of points between two models is within the noise.

### Metric definitions

- **Hit rate**: Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
- **Recall**: Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
- **MRR**: Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
- **nDCG**: Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
- **Precision**: Share of the top k results that are relevant. Low precision means more noise sent to the LLM.
