The best embedding model for French: RAG on French documents

French retrieval brings its own problems: missing accents, elisions, formal and informal registers, and administrative jargon. What to watch for, how to build a French test set, and a benchmark on French administrative and support documents.

  • embeddings
  • rag
  • french
  • multilingual
  • benchmark

Download the benchmark report (PDF)

French is one of the best-supported languages after English, so most multilingual embedding models handle it reasonably well. "Reasonably well" isn't good enough for a support bot or a public-service assistant, though, and the differences between models on French are larger than their English scores suggest.

This post covers what makes French retrieval different, how to test it, and which models to compare. The last section benchmarks eight models on French administrative guides and support articles.

What makes French different

Accents, often missing. Documents are written correctly ("délai", "résilier"), but users often type without accents ("delai", "resilier"), especially on mobile or on keyboards without French layouts. A good model treats both the same; a weaker one sees two different words.

Elision and contraction. "L'amende", "d'un", "qu'il", "au", "du": articles merge with the next word. Tokenizers split these in different ways, and models that saw little French may handle them poorly.

Register. Documents are formal ("Vous disposez d'un délai de 45 jours pour contester"), questions are often informal ("j'ai combien de temps pour contester une amende ?"). The model has to match the two.

Acronyms and jargon. French administration and business run on acronyms (CAF, URSSAF, APL, SIRET, TVA) that general training data barely covers, and on their own phrasing ("justificatif de domicile", "avis d'imposition").

Anglicisms and English terms. Tech and business documents mix in English words ("le dashboard", "un webhook", "le cloud"), sometimes alongside French equivalents ("tableau de bord").

Several Frenches. European French, Canadian French and African varieties differ in vocabulary ("courriel" vs "e-mail", "magasiner" vs "faire du shopping"). If your users are in Quebec, test with Quebec phrasing.

Build a French test set

  • Generate questions in French, not translated from English: translations sound unnatural and favor models that are good at translated text.
  • Include questions without accents and in an informal register, the way users type.
  • Add questions that use acronyms and others that spell out the full term.
  • If your corpus mixes French and English, add questions in each language.

Which models to test

  • Multilingual API models: Cohere Embed v4 and Embed Multilingual v3, Gemini Embedding, OpenAI text-embedding-3-large and 3-small.
  • Open-weight: Qwen3 Embedding, trained on many languages and self-hostable.
  • An English-only model as a baseline (such as BGE Base EN): it shows the gap a multilingual model closes.

French data is often personal or sensitive (administrative files, customer records). If GDPR or a client contract restricts where the data goes, check where each provider processes it, or consider a self-hosted open model.

The benchmark below runs eight models on French administrative guides and support articles. Look at the example questions, which are written the way French users actually ask. For what each metric in the tables means, see RAG evaluation metrics explained; for why a public leaderboard can't settle this for your documents, see why MTEB alone isn't enough.

Real benchmark - generated by Truvec

Benchmark: Embedding models on French administrative guides and help pages

Corpus
service-public.fr practical guides and French Wikipedia help pages
Documents
300
Chunks
4,345
Chunking
recursive, 1000 chars, 200 overlap
Golden set
100 questions, human-reviewed
Models
8 compared
Top k
5
Run
2026-11-14

Download the full report (PDF) Every question, with the passages each model retrieved.

Key results

  • Qwen3 Embedding 8B ranks first, with an MRR of 0.909 and a hit rate of 98.0% in the top 5.
  • One model is within 0.02 MRR of the winner (Cohere Embed v4): on this corpus they are practically tied.
  • Hit rate ranges from 75.0% to 100.0% across the 8 models.
  • Cheapest model within 2 points of the winner's hit rate: OpenAI text-embedding-3-small, $0.02 to embed this corpus against $0.10.
  • Fastest to embed a query: OpenAI text-embedding-3-small (7.1 ms per query, measured in batch from our servers).

Retrieval quality

Retrieval quality at top 5, ranked by MRR, then recall@k
ModelHit rate@5Recall@5MRRnDCG@5Precision@5
1Qwen3 Embedding 8Bfireworks - 4096 dims98.0%98.0%0.9090.92719.6%
2Cohere Embed v4cohere - 1536 dims99.0%99.0%0.8900.91519.8%
3Cohere Embed Multilingual v3cohere - 1024 dims99.0%99.0%0.8750.90419.8%
4Voyage 4 Largevoyage - 1024 dims97.0%97.0%0.8650.89119.4%
5OpenAI text-embedding-3-largeopenai - 3072 dims100.0%100.0%0.8470.88520.0%
6Gemini Embedding 2google - 3072 dims99.0%99.0%0.8370.87519.8%
7OpenAI text-embedding-3-smallopenai - 1536 dims97.0%97.0%0.8180.85619.4%
8BGE Base EN v1.5fireworks - 768 dims75.0%75.0%0.6120.64715.0%

Hit rate by position

How many questions the first results already answer. A model that finds the answer at position 1 lets you send fewer chunks to the LLM.

Hit rate by position: share of questions answered by the first j results
ModelTop 1Top 2Top 3Top 4Top 5
1Qwen3 Embedding 8Bfireworks - 4096 dims86.0%93.0%96.0%96.0%98.0%
2Cohere Embed v4cohere - 1536 dims82.0%92.0%96.0%97.0%99.0%
3Cohere Embed Multilingual v3cohere - 1024 dims79.0%91.0%97.0%98.0%99.0%
4Voyage 4 Largevoyage - 1024 dims79.0%90.0%94.0%95.0%97.0%
5OpenAI text-embedding-3-largeopenai - 3072 dims74.0%90.0%94.0%97.0%100.0%
6Gemini Embedding 2google - 3072 dims73.0%89.0%94.0%94.0%99.0%
7OpenAI text-embedding-3-smallopenai - 1536 dims73.0%82.0%91.0%94.0%97.0%
8BGE Base EN v1.5fireworks - 768 dims50.0%68.0%73.0%75.0%75.0%

Cost, speed and size

Embedding prices are the providers' list prices. Corpus cost is a one-time cost per model; the vector size is raw float32 storage, before the vector database's own overhead.

Cost, speed and size on this corpus
ModelPrice / 1M tokensEmbed the corpus1,000 queriesQuery embeddingSearch p95Vectors
1Qwen3 Embedding 8Bfireworks - 4096 dims$0.10$0.10$0.003110 ms60 ms67.9 MB
2Cohere Embed v4cohere - 1536 dims$0.12$0.11$0.003720 ms69 ms25.5 MB
3Cohere Embed Multilingual v3cohere - 1024 dims$0.10$0.10$0.00318.8 ms52 ms17.0 MB
4Voyage 4 Largevoyage - 1024 dims$0.12$0.11$0.003715 ms58 ms17.0 MB
5OpenAI text-embedding-3-largeopenai - 3072 dims$0.13$0.12$0.00409.0 ms90 ms50.9 MB
6Gemini Embedding 2google - 3072 dims$0.20$0.19$0.006220 ms59 ms50.9 MB
7OpenAI text-embedding-3-smallopenai - 1536 dims$0.02$0.02$0.00067.1 ms45 ms25.5 MB
8BGE Base EN v1.5fireworks - 768 dims$0.0080$0.0077$0.000219 ms51 ms12.7 MB

Score separation

Average similarity of the best relevant chunk and of the best irrelevant one. A wide gap makes a similarity threshold reliable; a narrow one means right and wrong chunks look alike to the model.

Score separation: how clearly each model scores the right chunk above the best wrong one
ModelRelevant chunkBest distractorGap
1Qwen3 Embedding 8Bfireworks - 4096 dims0.8290.7800.050
2Cohere Embed v4cohere - 1536 dims0.6720.6160.057
3Cohere Embed Multilingual v3cohere - 1024 dims0.7960.7560.041
4Voyage 4 Largevoyage - 1024 dims0.6370.5960.042
5OpenAI text-embedding-3-largeopenai - 3072 dims0.7290.6890.040
6Gemini Embedding 2google - 3072 dims0.8240.7980.025
7OpenAI text-embedding-3-smallopenai - 1536 dims0.7700.7360.033
8BGE Base EN v1.5fireworks - 768 dims0.7630.7430.019

Example questions

Rank of the first relevant chunk for each model; miss means it wasn't in the top 5. These are the questions where the models disagree most.

Example questions from the golden set
QuestionQwen3 Embedding 8BCohere Embed v4Cohere Embed Multilingual v3Voyage 4 LargeOpenAI text-embedding-3-largeGemini Embedding 2OpenAI text-embedding-3-smallBGE Base EN v1.5
Après la validation de la demande en ligne, quel délai faut-il généralement prévoir pour recevoir son permis de conduire ?
Expected passage
À partir de la réception du SMS vous informant de la validation de votre demande en ligne, il faut compter environ 15 jours pour recevoir votre permis de conduire (titre de conduite). À la fin de l’étape de fabrication de votre titre de conduite, vous recevez un mail contenant un
1miss1miss2543
Quelles sont les conditions pour utiliser un sas Parafe à l’arrivée en France ?
Expected passage
Vous n'êtes pas autorisé à utiliser un sas Parafe pour quitter le territoire. ## Qui peut utiliser le dispositif Parafe ? Votre passeport doit être biométrique et en cours de validité. Attention : Si vous vivez en France et voyagez seul à l'étranger, vous devez avoir une autorisa
542miss1531
Quels territoires sont concernés par une interdiction de retour lorsqu’une IRTF est prononcée en France ?
Expected passage
À savoir : Si vous êtes resté en France malgré l'OQTF, l'IRTF peut être prolongée de 2 ans au maximum dans la limite maximum de 5 ans. ## Quelles sont les conséquences de l'IRTF ? L'IRTF vous interdit de revenir non seulement en France, mais aussi dans tout pays de l'espace Schen
553255miss4
Que dois-je faire auprès de ma caisse de Sécurité sociale si ma situation professionnelle change, par exemple si je pars à la retraite ou change d'activité ?
Expected passage
En cas de changement de situation professionnelle(retraite, changement d'activité, etc.), contactez votre organisme de Sécurité sociale pour connaître les démarches à effectuer. En cas de changement d'organisme assurant votre prise en charge des frais de santé, l'ancien organisme
1544433miss
Où une victime peut-elle déposer plainte, et quels documents peut-elle obtenir à la fin de son audition ?
Expected passage
### Sur place Vous devez vous rendre dans un commissariat de police ou à la gendarmerie de votre choix. À savoir : Les services de police ou de gendarmerie sont obligés d'enregistrer votre plainte si vous êtes victime d'une infraction. Les officiers et agents de police judiciaire
322325missmiss
Comment faire enregistrer une modification de convention de Pacs et quelles pièces faut-il présenter en mairie ?
Expected passage
Ce document indique notamment vos identités et les références de votre Pacs initial. Vous devez utiliser le formulaire cerfa suivant (n° 15790) : ## Comment faire enregistrer la convention modificative de Pacs ? Vous devez faire enregistrer votre convention modificative de Pacs.
1122332miss
Comment un locataire peut-il contester un complément de loyer, et dans quel délai ?
Expected passage
Le montant du complément de loyer et les caractéristiques du logement qui le justifient doivent être indiqués dans le bail. Attention : Pour un bail signé depuis le 18 août 2022, le complément de loyer est interdit si le logement a au moins 1 de ces caractéristiques suivantes : s
2233121miss
Dans quelles situations un véhicule d’occasion peut-il circuler avec un certificat W garage ?
Expected passage
Leur apposition est obligatoire. Le numéro d’immatriculation est reproduit sur la plaque en caractères noirs sur fond rose. L'original du certificat doit être à bord du véhicule en circulation. Il doit être présenté en cas de contrôle. À noter : En cas d'utilisation du certificat
2131133miss

Notes

  • The corpus is 230 practical guides (fiches pratiques) for individuals from the service-public.fr open data, published under Licence Ouverte 2.0, and 70 help pages from French Wikipedia (Aide namespace, CC BY-SA 4.0), fetched on 14 November 2026. Guides were picked at random among those between 5,000 and 30,000 characters, spread across themes (justice, family, housing, money and taxes, work, transport, health, foreigners, papers and citizenship).
  • All 100 questions are written in French. 98 are about the administrative guides and 2 about the help pages: questions are drawn from chunks at random, and the guides make up most of the text.
  • BGE Base EN v1.5 is English-only: it is included as a baseline.
  • With 100 questions, a gap of a couple of points between two models is within the noise.

Methodology

Truvec split the documents into chunks, generated a golden set of questions with gpt-5.6-luna (each question is linked to the chunks that answer it), embedded the corpus and the questions with every model, and retrieved the top 5 chunks for each question by cosine similarity. Every model saw the same chunks and the same questions. Scores are averaged over all 100 questions.

Hit rate
Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
Recall
Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
MRR
Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
nDCG
Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
Precision
Share of the top k results that are relevant. Low precision means more noise sent to the LLM.

Downloads: full report (PDF) - benchmark.json - benchmark.csv - this post as Markdown

Qwen3 Embedding 8B won here, on these documents. Find the model that wins on yours: run the same benchmark on your documents with Truvec.