The best embedding model for French: RAG on French documents
French retrieval brings its own problems: missing accents, elisions, formal and informal registers, and administrative jargon. What to watch for, how to build a French test set, and a benchmark on French administrative and support documents.
French is one of the best-supported languages after English, so most multilingual embedding models handle it reasonably well. "Reasonably well" isn't good enough for a support bot or a public-service assistant, though, and the differences between models on French are larger than their English scores suggest.
This post covers what makes French retrieval different, how to test it, and which models to compare. The last section benchmarks eight models on French administrative guides and support articles.
What makes French different
Accents, often missing. Documents are written correctly ("délai", "résilier"), but users often type without accents ("delai", "resilier"), especially on mobile or on keyboards without French layouts. A good model treats both the same; a weaker one sees two different words.
Elision and contraction. "L'amende", "d'un", "qu'il", "au", "du": articles merge with the next word. Tokenizers split these in different ways, and models that saw little French may handle them poorly.
Register. Documents are formal ("Vous disposez d'un délai de 45 jours pour contester"), questions are often informal ("j'ai combien de temps pour contester une amende ?"). The model has to match the two.
Acronyms and jargon. French administration and business run on acronyms (CAF, URSSAF, APL, SIRET, TVA) that general training data barely covers, and on their own phrasing ("justificatif de domicile", "avis d'imposition").
Anglicisms and English terms. Tech and business documents mix in English words ("le dashboard", "un webhook", "le cloud"), sometimes alongside French equivalents ("tableau de bord").
Several Frenches. European French, Canadian French and African varieties differ in vocabulary ("courriel" vs "e-mail", "magasiner" vs "faire du shopping"). If your users are in Quebec, test with Quebec phrasing.
Build a French test set
- Generate questions in French, not translated from English: translations sound unnatural and favor models that are good at translated text.
- Include questions without accents and in an informal register, the way users type.
- Add questions that use acronyms and others that spell out the full term.
- If your corpus mixes French and English, add questions in each language.
Which models to test
- Multilingual API models: Cohere Embed v4 and Embed Multilingual v3, Gemini Embedding, OpenAI text-embedding-3-large and 3-small.
- Open-weight: Qwen3 Embedding, trained on many languages and self-hostable.
- An English-only model as a baseline (such as BGE Base EN): it shows the gap a multilingual model closes.
French data is often personal or sensitive (administrative files, customer records). If GDPR or a client contract restricts where the data goes, check where each provider processes it, or consider a self-hosted open model.
The benchmark below runs eight models on French administrative guides and support articles. Look at the example questions, which are written the way French users actually ask. For what each metric in the tables means, see RAG evaluation metrics explained; for why a public leaderboard can't settle this for your documents, see why MTEB alone isn't enough.
Real benchmark - generated by Truvec
Benchmark: Embedding models on French administrative guides and help pages
- Documents
- 300
- Chunks
- 4,345
- Chunking
- recursive, 1000 chars, 200 overlap
- Golden set
- 100 questions, human-reviewed
- Models
- 8 compared
- Top k
- 5
- Run
- 2026-11-14
Download the full report (PDF) Every question, with the passages each model retrieved.
Key results
- Qwen3 Embedding 8B ranks first, with an MRR of 0.909 and a hit rate of 98.0% in the top 5.
- One model is within 0.02 MRR of the winner (Cohere Embed v4): on this corpus they are practically tied.
- Hit rate ranges from 75.0% to 100.0% across the 8 models.
- Cheapest model within 2 points of the winner's hit rate: OpenAI text-embedding-3-small, $0.02 to embed this corpus against $0.10.
- Fastest to embed a query: OpenAI text-embedding-3-small (7.1 ms per query, measured in batch from our servers).
Retrieval quality
| Model | Hit rate@5 | Recall@5 | MRR | nDCG@5 | Precision@5 |
|---|---|---|---|---|---|
| 1Qwen3 Embedding 8B | 98.0% | 98.0% | 0.927 | 19.6% | |
| 2Cohere Embed v4 | 99.0% | 99.0% | 0.915 | 19.8% | |
| 3Cohere Embed Multilingual v3 | 99.0% | 99.0% | 0.904 | 19.8% | |
| 4Voyage 4 Large | 97.0% | 97.0% | 0.891 | 19.4% | |
| 5OpenAI text-embedding-3-large | 100.0% | 100.0% | 0.885 | 20.0% | |
| 6Gemini Embedding 2 | 99.0% | 99.0% | 0.875 | 19.8% | |
| 7OpenAI text-embedding-3-small | 97.0% | 97.0% | 0.856 | 19.4% | |
| 8BGE Base EN v1.5 | 75.0% | 75.0% | 0.647 | 15.0% |
Hit rate by position
How many questions the first results already answer. A model that finds the answer at position 1 lets you send fewer chunks to the LLM.
| Model | Top 1 | Top 2 | Top 3 | Top 4 | Top 5 |
|---|---|---|---|---|---|
| 1Qwen3 Embedding 8B | 86.0% | 93.0% | 96.0% | 96.0% | 98.0% |
| 2Cohere Embed v4 | 82.0% | 92.0% | 96.0% | 97.0% | 99.0% |
| 3Cohere Embed Multilingual v3 | 79.0% | 91.0% | 97.0% | 98.0% | 99.0% |
| 4Voyage 4 Large | 79.0% | 90.0% | 94.0% | 95.0% | 97.0% |
| 5OpenAI text-embedding-3-large | 74.0% | 90.0% | 94.0% | 97.0% | 100.0% |
| 6Gemini Embedding 2 | 73.0% | 89.0% | 94.0% | 94.0% | 99.0% |
| 7OpenAI text-embedding-3-small | 73.0% | 82.0% | 91.0% | 94.0% | 97.0% |
| 8BGE Base EN v1.5 | 50.0% | 68.0% | 73.0% | 75.0% | 75.0% |
Cost, speed and size
Embedding prices are the providers' list prices. Corpus cost is a one-time cost per model; the vector size is raw float32 storage, before the vector database's own overhead.
| Model | Price / 1M tokens | Embed the corpus | 1,000 queries | Query embedding | Search p95 | Vectors |
|---|---|---|---|---|---|---|
| 1Qwen3 Embedding 8B | $0.10 | $0.10 | $0.0031 | 10 ms | 60 ms | 67.9 MB |
| 2Cohere Embed v4 | $0.12 | $0.11 | $0.0037 | 20 ms | 69 ms | 25.5 MB |
| 3Cohere Embed Multilingual v3 | $0.10 | $0.10 | $0.0031 | 8.8 ms | 52 ms | 17.0 MB |
| 4Voyage 4 Large | $0.12 | $0.11 | $0.0037 | 15 ms | 58 ms | 17.0 MB |
| 5OpenAI text-embedding-3-large | $0.13 | $0.12 | $0.0040 | 9.0 ms | 90 ms | 50.9 MB |
| 6Gemini Embedding 2 | $0.20 | $0.19 | $0.0062 | 20 ms | 59 ms | 50.9 MB |
| 7OpenAI text-embedding-3-small | $0.02 | $0.02 | $0.0006 | 7.1 ms | 45 ms | 25.5 MB |
| 8BGE Base EN v1.5 | $0.0080 | $0.0077 | $0.0002 | 19 ms | 51 ms | 12.7 MB |
Score separation
Average similarity of the best relevant chunk and of the best irrelevant one. A wide gap makes a similarity threshold reliable; a narrow one means right and wrong chunks look alike to the model.
| Model | Relevant chunk | Best distractor | Gap |
|---|---|---|---|
| 1Qwen3 Embedding 8B | 0.829 | 0.780 | 0.050 |
| 2Cohere Embed v4 | 0.672 | 0.616 | 0.057 |
| 3Cohere Embed Multilingual v3 | 0.796 | 0.756 | 0.041 |
| 4Voyage 4 Large | 0.637 | 0.596 | 0.042 |
| 5OpenAI text-embedding-3-large | 0.729 | 0.689 | 0.040 |
| 6Gemini Embedding 2 | 0.824 | 0.798 | 0.025 |
| 7OpenAI text-embedding-3-small | 0.770 | 0.736 | 0.033 |
| 8BGE Base EN v1.5 | 0.763 | 0.743 | 0.019 |
Example questions
Rank of the first relevant chunk for each model; miss means it wasn't in the top 5. These are the questions where the models disagree most.
| Question | Qwen3 Embedding 8B | Cohere Embed v4 | Cohere Embed Multilingual v3 | Voyage 4 Large | OpenAI text-embedding-3-large | Gemini Embedding 2 | OpenAI text-embedding-3-small | BGE Base EN v1.5 |
|---|---|---|---|---|---|---|---|---|
Après la validation de la demande en ligne, quel délai faut-il généralement prévoir pour recevoir son permis de conduire ?Expected passageÀ partir de la réception du SMS vous informant de la validation de votre demande en ligne, il faut compter environ 15 jours pour recevoir votre permis de conduire (titre de conduite). À la fin de l’étape de fabrication de votre titre de conduite, vous recevez un mail contenant un | 1 | miss | 1 | miss | 2 | 5 | 4 | 3 |
Quelles sont les conditions pour utiliser un sas Parafe à l’arrivée en France ?Expected passageVous n'êtes pas autorisé à utiliser un sas Parafe pour quitter le territoire. ## Qui peut utiliser le dispositif Parafe ? Votre passeport doit être biométrique et en cours de validité. Attention : Si vous vivez en France et voyagez seul à l'étranger, vous devez avoir une autorisa | 5 | 4 | 2 | miss | 1 | 5 | 3 | 1 |
Quels territoires sont concernés par une interdiction de retour lorsqu’une IRTF est prononcée en France ?Expected passageÀ savoir : Si vous êtes resté en France malgré l'OQTF, l'IRTF peut être prolongée de 2 ans au maximum dans la limite maximum de 5 ans. ## Quelles sont les conséquences de l'IRTF ? L'IRTF vous interdit de revenir non seulement en France, mais aussi dans tout pays de l'espace Schen | 5 | 5 | 3 | 2 | 5 | 5 | miss | 4 |
Que dois-je faire auprès de ma caisse de Sécurité sociale si ma situation professionnelle change, par exemple si je pars à la retraite ou change d'activité ?Expected passageEn cas de changement de situation professionnelle(retraite, changement d'activité, etc.), contactez votre organisme de Sécurité sociale pour connaître les démarches à effectuer. En cas de changement d'organisme assurant votre prise en charge des frais de santé, l'ancien organisme | 1 | 5 | 4 | 4 | 4 | 3 | 3 | miss |
Où une victime peut-elle déposer plainte, et quels documents peut-elle obtenir à la fin de son audition ?Expected passage### Sur place Vous devez vous rendre dans un commissariat de police ou à la gendarmerie de votre choix. À savoir : Les services de police ou de gendarmerie sont obligés d'enregistrer votre plainte si vous êtes victime d'une infraction. Les officiers et agents de police judiciaire | 3 | 2 | 2 | 3 | 2 | 5 | miss | miss |
Comment faire enregistrer une modification de convention de Pacs et quelles pièces faut-il présenter en mairie ?Expected passageCe document indique notamment vos identités et les références de votre Pacs initial. Vous devez utiliser le formulaire cerfa suivant (n° 15790) : ## Comment faire enregistrer la convention modificative de Pacs ? Vous devez faire enregistrer votre convention modificative de Pacs. | 1 | 1 | 2 | 2 | 3 | 3 | 2 | miss |
Comment un locataire peut-il contester un complément de loyer, et dans quel délai ?Expected passageLe montant du complément de loyer et les caractéristiques du logement qui le justifient doivent être indiqués dans le bail. Attention : Pour un bail signé depuis le 18 août 2022, le complément de loyer est interdit si le logement a au moins 1 de ces caractéristiques suivantes : s | 2 | 2 | 3 | 3 | 1 | 2 | 1 | miss |
Dans quelles situations un véhicule d’occasion peut-il circuler avec un certificat W garage ?Expected passageLeur apposition est obligatoire. Le numéro d’immatriculation est reproduit sur la plaque en caractères noirs sur fond rose. L'original du certificat doit être à bord du véhicule en circulation. Il doit être présenté en cas de contrôle. À noter : En cas d'utilisation du certificat | 2 | 1 | 3 | 1 | 1 | 3 | 3 | miss |
Notes
- The corpus is 230 practical guides (fiches pratiques) for individuals from the service-public.fr open data, published under Licence Ouverte 2.0, and 70 help pages from French Wikipedia (Aide namespace, CC BY-SA 4.0), fetched on 14 November 2026. Guides were picked at random among those between 5,000 and 30,000 characters, spread across themes (justice, family, housing, money and taxes, work, transport, health, foreigners, papers and citizenship).
- All 100 questions are written in French. 98 are about the administrative guides and 2 about the help pages: questions are drawn from chunks at random, and the guides make up most of the text.
- BGE Base EN v1.5 is English-only: it is included as a baseline.
- With 100 questions, a gap of a couple of points between two models is within the noise.
Methodology
Truvec split the documents into chunks, generated a golden set of questions with gpt-5.6-luna (each question is linked to the chunks that answer it), embedded the corpus and the questions with every model, and retrieved the top 5 chunks for each question by cosine similarity. Every model saw the same chunks and the same questions. Scores are averaged over all 100 questions.
- Hit rate
- Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
- Recall
- Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
- MRR
- Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
- nDCG
- Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
- Precision
- Share of the top k results that are relevant. Low precision means more noise sent to the LLM.
Downloads: full report (PDF) - benchmark.json - benchmark.csv - this post as Markdown
Qwen3 Embedding 8B won here, on these documents. Find the model that wins on yours: run the same benchmark on your documents with Truvec.