The best embedding model for legal documents: contracts, clauses and regulations

Legal text is hard for embedding models: long clauses, defined terms, cross-references and near-identical boilerplate. What makes legal retrieval different, how to chunk and test it, and a benchmark on contracts and regulations.

  • embeddings
  • rag
  • legal
  • benchmark

Download the benchmark report (PDF)

Legal RAG has to be precise. A lawyer asking whether a contract can be terminated without cause needs the termination clause of that contract, not a similar clause from another agreement and not the definition of "Term". Embedding models built and benchmarked on web text struggle with exactly what makes legal documents legal.

This post covers what makes legal retrieval hard, how to prepare legal documents for embedding, and how to test models on them. The last section is a benchmark on public contracts and regulations.

Boilerplate everywhere. Contracts reuse the same clauses with small changes: a liability cap of 12 months of fees in one agreement, of the total contract value in another. Embeddings place near-identical clauses next to each other, so the right clause competes with many convincing look-alikes. These are the hardest distractors there are.

Meaning hides in small words. "Shall", "may", "unless", "except as provided in Section 9" change the meaning of a clause completely, but barely change its embedding. A model can retrieve the right topic and still return the clause that says the opposite.

Defined terms. "Confidential Information", "Services" and "Affiliate" are defined once, then used everywhere with their capital letter and their precise meaning. Questions use everyday words ("secret data", "the work", "sister company"), so the model has to connect the two.

Cross-references. "Subject to Section 12.3" sends the reader elsewhere. A chunk containing that reference is incomplete on its own, and a question about Section 12.3 by number needs exact matching, which embeddings are bad at.

Length. Clauses run long, and regulations nest articles, paragraphs and points. Short-input models (512 tokens) may truncate a clause before the part that matters.

Prepare the documents

  • Chunk by structure, not by size. Split on articles, sections and clauses. A chunk should be a complete clause with its heading ("12. Limitation of Liability"), so the heading's words help retrieval.
  • Use larger chunks than for prose. Legal clauses need their full text to be understood. Test chunk sizes around 1,500 to 2,500 characters, and check which models truncate them.
  • Keep identity in metadata. Store the document, party, date and jurisdiction with each chunk, and filter on them. No embedding model reliably tells two versions of the same NDA apart.
  • Add keyword search for references. Questions like "what does Section 14.2 say?" or "Article 33 GDPR" are best answered by exact matching. Hybrid search (keywords plus embeddings) covers both.

Test the models on what matters

A legal golden set should include:

  • questions phrased in plain language about defined terms;
  • questions whose answer is an exception ("can the supplier subcontract?" when the rule is "no, except with written consent");
  • questions that only one of several similar contracts answers;
  • a few questions where the right answer is "this contract doesn't say".

Look at precision as well as hit rate: in legal work, a wrong clause in the context can be worse than a missing one, because the LLM may cite it with confidence.

Confidentiality and hosting

Legal documents are often confidential, and sending them to an embedding API may be ruled out by a client agreement or a regulator. Open-weight models (Qwen3 Embedding, BGE, GTE, mxbai, Nomic) can run in your own infrastructure. Include at least one of them in your comparison: you'll see exactly what self-hosting costs you in retrieval quality, if anything.

What to take from the benchmark

The benchmark below uses public contracts, terms of service and regulations. Your documents will differ: a firm's M&A contracts are not consumer terms of service. Look at the gaps between models, the score separation (how clearly each model sets the right clause apart from look-alikes), and the example questions. Then run the same comparison on a sample of your own documents before you choose.

Real benchmark - generated by Truvec

Benchmark: Embedding models on contracts and regulations

Corpus
Commercial contracts (CUAD), GitHub terms and policies, and three EU regulations
Documents
58
Chunks
5,024
Chunking
recursive, 1000 chars, 200 overlap
Golden set
100 questions, human-reviewed
Models
8 compared
Top k
5
Run
2026-11-14

Download the full report (PDF) Every question, with the passages each model retrieved.

Key results

  • Voyage 4 Large ranks first, with an MRR of 0.828 and a hit rate of 94.0% in the top 5.
  • One model is within 0.02 MRR of the winner (Voyage 4): on this corpus they are practically tied.
  • Hit rate ranges from 77.0% to 94.0% across the 8 models.
  • Cheapest model within 2 points of the winner's hit rate: Voyage 4, $0.05 to embed this corpus against $0.09.
  • Fastest to embed a query: OpenAI text-embedding-3-small (6.0 ms per query, measured in batch from our servers).

Retrieval quality

Retrieval quality at top 5, ranked by MRR, then recall@k
ModelHit rate@5Recall@5MRRnDCG@5Precision@5
1Voyage 4 Largevoyage - 1024 dims94.0%94.0%0.8280.85619.2%
2Voyage 4voyage - 1024 dims92.0%91.5%0.8150.83818.6%
3Gemini Embedding 2google - 3072 dims88.0%87.5%0.7420.77317.8%
4Cohere Embed v4cohere - 1536 dims84.0%83.5%0.7340.75717.0%
5Qwen3 Embedding 8Bfireworks - 4096 dims86.0%86.0%0.7050.74417.6%
6OpenAI text-embedding-3-largeopenai - 3072 dims83.0%83.0%0.6970.72917.0%
7OpenAI text-embedding-3-smallopenai - 1536 dims77.0%76.5%0.6560.68015.6%
8BGE Base EN v1.5fireworks - 768 dims78.0%77.5%0.6190.65415.8%

Hit rate by position

How many questions the first results already answer. A model that finds the answer at position 1 lets you send fewer chunks to the LLM.

Hit rate by position: share of questions answered by the first j results
ModelTop 1Top 2Top 3Top 4Top 5
1Voyage 4 Largevoyage - 1024 dims75.0%87.0%89.0%92.0%94.0%
2Voyage 4voyage - 1024 dims73.0%88.0%89.0%90.0%92.0%
3Gemini Embedding 2google - 3072 dims65.0%77.0%84.0%85.0%88.0%
4Cohere Embed v4cohere - 1536 dims67.0%75.0%79.0%81.0%84.0%
5Qwen3 Embedding 8Bfireworks - 4096 dims61.0%71.0%81.0%85.0%86.0%
6OpenAI text-embedding-3-largeopenai - 3072 dims60.0%75.0%78.0%82.0%83.0%
7OpenAI text-embedding-3-smallopenai - 1536 dims57.0%70.0%74.0%77.0%77.0%
8BGE Base EN v1.5fireworks - 768 dims52.0%65.0%68.0%75.0%78.0%

Cost, speed and size

Embedding prices are the providers' list prices. Corpus cost is a one-time cost per model; the vector size is raw float32 storage, before the vector database's own overhead.

Cost, speed and size on this corpus
ModelPrice / 1M tokensEmbed the corpus1,000 queriesQuery embeddingSearch p95Vectors
1Voyage 4 Largevoyage - 1024 dims$0.12$0.09$0.002613 ms52 ms19.6 MB
2Voyage 4voyage - 1024 dims$0.06$0.05$0.001319 ms23 ms19.6 MB
3Gemini Embedding 2google - 3072 dims$0.20$0.15$0.004319 ms44 ms58.9 MB
4Cohere Embed v4cohere - 1536 dims$0.12$0.09$0.002613 ms27 ms29.4 MB
5Qwen3 Embedding 8Bfireworks - 4096 dims$0.10$0.08$0.0021166 ms72 ms78.5 MB
6OpenAI text-embedding-3-largeopenai - 3072 dims$0.13$0.10$0.00286.6 ms20 ms58.9 MB
7OpenAI text-embedding-3-smallopenai - 1536 dims$0.02$0.02$0.00046.0 ms32 ms29.4 MB
8BGE Base EN v1.5fireworks - 768 dims$0.0080$0.0062$0.000214 ms23 ms14.7 MB

Score separation

Average similarity of the best relevant chunk and of the best irrelevant one. A wide gap makes a similarity threshold reliable; a narrow one means right and wrong chunks look alike to the model.

Score separation: how clearly each model scores the right chunk above the best wrong one
ModelRelevant chunkBest distractorGap
1Voyage 4 Largevoyage - 1024 dims0.5760.5060.068
2Voyage 4voyage - 1024 dims0.6280.5610.066
3Gemini Embedding 2google - 3072 dims0.8070.7830.024
4Cohere Embed v4cohere - 1536 dims0.6680.6210.047
5Qwen3 Embedding 8Bfireworks - 4096 dims0.7810.7460.033
6OpenAI text-embedding-3-largeopenai - 3072 dims0.6800.6340.049
7OpenAI text-embedding-3-smallopenai - 1536 dims0.7120.6710.040
8BGE Base EN v1.5fireworks - 768 dims0.7610.7440.020

Example questions

Rank of the first relevant chunk for each model; miss means it wasn't in the top 5. These are the questions where the models disagree most.

Example questions from the golden set
QuestionVoyage 4 LargeVoyage 4Gemini Embedding 2Cohere Embed v4Qwen3 Embedding 8BOpenAI text-embedding-3-largeOpenAI text-embedding-3-smallBGE Base EN v1.5
Under the Whitesmoke promotion and distribution agreement, what restrictions apply to the Distributor's treatment of products already installed by end users after the agreement ends?
Expected passage
3.8 [ * ]. During the Term and for a period of [ * ] following the expiration or termination of this Agreement, Distributor shall not, and shall not engage any third party to: (a) restrict, modify, or reconfigure in any manner any of the Products that have been installed by End U
11232miss44
What responsibilities apply to anyone collecting personal information through the GitHub Service?
Expected passage
Your use of information from the Service must comply with the GitHub Privacy Statement. ## 8. Privacy Misuse of personal information is prohibited. Any person, entity, or service collecting data from the Service must comply with the GitHub Privacy Statement, particularly in regar
22312missmiss4
Under the video-on-demand content license between Rogers and EuroMedia, when may a party disclose the other's confidential information to comply with legal requirements, and what must it do beforehand?
Expected passage
to the greatest extent possible. (b) Each party agrees to use the other party's confidential information solely for the purpose of fulfilling its obligations under this Agreement, to hold the other party's confidential information in confidence and to protect the confidentiality
52453miss44
Under GitHub's Corporate Terms of Service, when can GitHub assign the agreement without the customer's consent?
Expected passage
### 3. Non-Assignability Neither Party may assign or otherwise transfer this Agreement, in whole or in part, without the other Party's prior written consent, such consent not to be unreasonably withheld, and any attempt to do so will be null and void, except that GitHub may assig
111132miss4
What kinds of conduct are prohibited at the GitHub event?
Expected passage
Unacceptable behavior includes, but is not limited to: * The use of sexualized language or imagery * Inappropriate physical contact, sexual attention, or advances * Trolling, insulting or derogatory comments or behavior * Personal or political attacks toward Event Participants *
11223115
What extra charges does a sponsor pay under the program, including when using a credit card?
Expected passage
### 2.2. Fees. An additional service fee ("Service Fee") of 3% per transaction will be deducted from Sponsor's account to cover GitHub's administrative costs associated with offering the Program. If payment is made via credit card, an additional 3% fee will be applied. ### 2.3. T
1111miss4miss3
What uses of GitHub-hosted Actions are not allowed?
Expected passage
* The provision of a stand-alone or integrated application or service offering the Actions product or service, or any elements of the Actions product or service, for commercial purposes; * Any activity that places a burden on our servers, where that burden is disproportionate to
1112322miss
Do I need GitHub's permission to use its trademarks or logos?
Expected passage
## GitHub's Logos The license grant included with the Software is not for GitHub's trademarks, which include the Software logo designs. GitHub reserves all trademark and copyright rights in and to all GitHub trademarks. GitHub's logos include, for instance, the stylized designs t
11314211

Notes

  • The corpus has three parts. 36 commercial contracts from the CUAD dataset (The Atticus Project, CC BY 4.0, text extracted from SEC EDGAR filings), picked across contract types among those between 25,000 and 90,000 characters. 19 terms, agreements and policies from github.com/github/site-policy (CC0). The English texts of Regulation (EU) 2016/679 (GDPR), Regulation (EU) 2022/2065 (Digital Services Act) and Regulation (EU) 2024/1689 (AI Act), from the Publications Office of the EU.
  • The contract texts are used as published in CUAD: page numbers, redaction marks and layout left over from the filings are still in the chunks.
  • Reviewing the questions, we rewrote 27 that did not say which document they were about ("Which state’s law governs the agreement?") to name the contract or its parties, the way a lawyer would ask. The name is often absent from the chunk that holds the clause, which is part of what makes legal retrieval hard.
  • With 100 questions, a gap of a couple of points between two models is within the noise.

Methodology

Truvec split the documents into chunks, generated a golden set of questions with gpt-5.6-luna (each question is linked to the chunks that answer it), embedded the corpus and the questions with every model, and retrieved the top 5 chunks for each question by cosine similarity. Every model saw the same chunks and the same questions. Scores are averaged over all 100 questions.

Hit rate
Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
Recall
Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
MRR
Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
nDCG
Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
Precision
Share of the top k results that are relevant. Low precision means more noise sent to the LLM.

Downloads: full report (PDF) - benchmark.json - benchmark.csv - this post as Markdown

Voyage 4 Large won here, on these documents. Find the model that wins on yours: run the same benchmark on your documents with Truvec.