The best multilingual embedding model for RAG
Multilingual RAG means questions and documents in several languages, sometimes different ones. What multilingual retrieval needs, the pitfalls of language bias and tokenization, how to test per language, and a benchmark on a five-language help center.
Most embedding benchmarks are in English, and most RAG tutorials assume English documents. Many real systems aren't: a help center in five languages, a European company's internal docs, a support bot that answers in the user's language. Multilingual retrieval has its own failure modes, and a model's English score says little about them.
This post explains what a multilingual RAG system needs from its embedding model and how to test it. The last section benchmarks the main multilingual models on a help center in English, French, Spanish, German and Arabic.
Three multilingual situations
Same language, several languages. Each user asks in their language, and the answer exists in that language. The model needs to be good in every language you serve, not just on average.
Cross-lingual. The user asks in French, the answer only exists in English. The model must map both languages into the same space, so that a French question lands near an English passage. Not all multilingual models do this well.
Mixed text. Real documents mix languages: English product names in French articles, code in Arabic tutorials, quotes in another language. The model must not be thrown off by the switch.
Know which situations you have before you test: the right model for one isn't always the right model for the others.
Pitfalls
Language bias. Many models prefer passages in the same language as the question, even when a passage in another language answers it better. In a mixed corpus, this means a French question retrieves French passages that are only roughly relevant, over the exact answer in English.
Uneven quality across languages. A model can be excellent in English and Spanish and weak in Arabic or Thai. Averages hide this: always look at results per language.
Tokenization cost. Tokenizers built mostly on English text split other languages into more tokens. The same text can cost noticeably more to embed in Arabic or Hindi than in English, and long texts may hit input limits sooner.
English-only models. Models such as BGE Base EN are trained on English. On other languages they return poor results that look plausible at first glance.
How to test
- Build the golden set per language, in proportion to your real traffic. Generated questions should be in the language users actually ask in.
- Add cross-lingual questions if users may ask in a language the answer isn't written in.
- Compare per language before you look at the overall ranking. A model that wins overall but fails in one language you serve is the wrong choice.
- Include an English-only model as a baseline. It shows how much multilingual support is worth on your data.
Candidates
Cohere's multilingual models (Embed v4, Embed Multilingual v3), Google's Gemini embeddings, Qwen3 Embedding and OpenAI text-embedding-3-large are the usual shortlist. Voyage's models are worth including as well. Check each provider's documentation for the languages it supports.
The benchmark below runs eight models on a five-language help center, with an English-only model as a baseline. Look at the example questions in each language: averages hide which languages each model struggles with. For what each metric in the tables means, see RAG evaluation metrics explained; for why a public leaderboard can't settle this for your documents, see why MTEB alone isn't enough.
Real benchmark - generated by Truvec
Benchmark: Multilingual embedding models on a 5-language help center
- Documents
- 165
- Chunks
- 2,050
- Chunking
- recursive, 1000 chars, 200 overlap
- Golden set
- 100 questions, human-reviewed
- Models
- 8 compared
- Top k
- 5
- Run
- 2026-11-14
Download the full report (PDF) Every question, with the passages each model retrieved.
Key results
- OpenAI text-embedding-3-large ranks first, with an MRR of 0.848 and a hit rate of 97.0% in the top 5.
- It leads the runner-up, Cohere Embed Multilingual v3, by 0.037 MRR.
- Hit rate ranges from 70.0% to 97.0% across the 8 models.
- Cheapest model within 2 points of the winner's hit rate: Cohere Embed v4, $0.05 to embed this corpus against $0.06.
- Fastest to embed a query: OpenAI text-embedding-3-small (5.7 ms per query, measured in batch from our servers).
Retrieval quality
| Model | Hit rate@5 | Recall@5 | MRR | nDCG@5 | Precision@5 |
|---|---|---|---|---|---|
| 1OpenAI text-embedding-3-large | 97.0% | 96.5% | 0.876 | 19.6% | |
| 2Cohere Embed Multilingual v3 | 93.0% | 92.5% | 0.839 | 18.8% | |
| 3Cohere Embed v4 | 95.0% | 94.5% | 0.831 | 19.2% | |
| 4Gemini Embedding 2 | 93.0% | 92.5% | 0.815 | 18.6% | |
| 5Voyage 4 Large | 95.0% | 95.0% | 0.823 | 19.2% | |
| 6Qwen3 Embedding 8B | 90.0% | 89.5% | 0.785 | 18.0% | |
| 7OpenAI text-embedding-3-small | 88.0% | 88.0% | 0.768 | 18.0% | |
| 8BGE Base EN v1.5 | 70.0% | 70.0% | 0.610 | 14.4% |
Hit rate by position
How many questions the first results already answer. A model that finds the answer at position 1 lets you send fewer chunks to the LLM.
| Model | Top 1 | Top 2 | Top 3 | Top 4 | Top 5 |
|---|---|---|---|---|---|
| 1OpenAI text-embedding-3-large | 78.0% | 85.0% | 91.0% | 93.0% | 97.0% |
| 2Cohere Embed Multilingual v3 | 74.0% | 82.0% | 87.0% | 91.0% | 93.0% |
| 3Cohere Embed v4 | 70.0% | 81.0% | 88.0% | 90.0% | 95.0% |
| 4Gemini Embedding 2 | 68.0% | 83.0% | 86.0% | 91.0% | 93.0% |
| 5Voyage 4 Large | 68.0% | 79.0% | 88.0% | 91.0% | 95.0% |
| 6Qwen3 Embedding 8B | 65.0% | 79.0% | 83.0% | 86.0% | 90.0% |
| 7OpenAI text-embedding-3-small | 64.0% | 76.0% | 83.0% | 84.0% | 88.0% |
| 8BGE Base EN v1.5 | 52.0% | 59.0% | 62.0% | 66.0% | 70.0% |
Cost, speed and size
Embedding prices are the providers' list prices. Corpus cost is a one-time cost per model; the vector size is raw float32 storage, before the vector database's own overhead.
| Model | Price / 1M tokens | Embed the corpus | 1,000 queries | Query embedding | Search p95 | Vectors |
|---|---|---|---|---|---|---|
| 1OpenAI text-embedding-3-large | $0.13 | $0.06 | $0.0039 | 8.1 ms | 54 ms | 24.0 MB |
| 2Cohere Embed Multilingual v3 | $0.10 | $0.04 | $0.0030 | 7.8 ms | 49 ms | 8.0 MB |
| 3Cohere Embed v4 | $0.12 | $0.05 | $0.0036 | 11 ms | 31 ms | 12.0 MB |
| 4Gemini Embedding 2 | $0.20 | $0.09 | $0.0060 | 22 ms | 65 ms | 24.0 MB |
| 5Voyage 4 Large | $0.12 | $0.05 | $0.0036 | 17 ms | 33 ms | 8.0 MB |
| 6Qwen3 Embedding 8B | $0.10 | $0.04 | $0.0030 | 21 ms | 56 ms | 32.0 MB |
| 7OpenAI text-embedding-3-small | $0.02 | $0.0087 | $0.0006 | 5.7 ms | 52 ms | 12.0 MB |
| 8BGE Base EN v1.5 | $0.0080 | $0.0032 | $0.0002 | 18 ms | 25 ms | 6.0 MB |
Score separation
Average similarity of the best relevant chunk and of the best irrelevant one. A wide gap makes a similarity threshold reliable; a narrow one means right and wrong chunks look alike to the model.
| Model | Relevant chunk | Best distractor | Gap |
|---|---|---|---|
| 1OpenAI text-embedding-3-large | 0.660 | 0.621 | 0.039 |
| 2Cohere Embed Multilingual v3 | 0.730 | 0.696 | 0.035 |
| 3Cohere Embed v4 | 0.628 | 0.599 | 0.027 |
| 4Gemini Embedding 2 | 0.813 | 0.800 | 0.012 |
| 5Voyage 4 Large | 0.616 | 0.602 | 0.013 |
| 6Qwen3 Embedding 8B | 0.805 | 0.784 | 0.019 |
| 7OpenAI text-embedding-3-small | 0.693 | 0.648 | 0.038 |
| 8BGE Base EN v1.5 | 0.761 | 0.728 | 0.033 |
Example questions
Rank of the first relevant chunk for each model; miss means it wasn't in the top 5. These are the questions where the models disagree most.
| Question | OpenAI text-embedding-3-large | Cohere Embed Multilingual v3 | Cohere Embed v4 | Gemini Embedding 2 | Voyage 4 Large | Qwen3 Embedding 8B | OpenAI text-embedding-3-small | BGE Base EN v1.5 |
|---|---|---|---|---|---|---|---|---|
¿Cómo decide WebFonts si usa una fuente de mi dispositivo, la descarga o la recupera de la caché?Expected passage## Uso Cuando la extensión está instalada y habilitada, tras el inicio de sesión, las preferencias de enlace en la página wiki, aparecerá un menú para seleccionar una fuente de la página. De forma predeterminada, la primera fuente de dicho menú se aplica a la wiki. Un usuario pue | 1 | 1 | 1 | 4 | 3 | 5 | 2 | miss |
¿Qué permiso se necesita para crear listas de envío de MassMessage, y dónde se puede crear una lista nueva?Expected passageNote that creating delivery lists requires the user to have the createwithcontentmodel user right (see phab:T92795). On most Wikimedia projects, only logged-in users have createwithcontentmodel rights. MassMessage includes a special page for managing delivery lists. To create a n | 5 | miss | 4 | 3 | 3 | 2 | 3 | miss |
Quelles sont les étapes pour commencer une nouvelle traduction avec l'outil de traduction de contenu ?Expected passageDeuxième manière: cliquez sur "Contributions", puis sur le bouton "Traduction" en haut de la page. Troisième méthode: chercher la page à traduire sur le wiki dans la langue d'origine, et cliquez sur la langue de destination dans la liste des liens inter-langage de la barre vertic | 3 | 3 | miss | 2 | 1 | 5 | 5 | miss |
كيف أشترك في موضوع نقاش ثم أوقف تلقي الإشعارات عنه لاحقًا؟Expected passagepreloadtitle – لو ذكرت، ستستخدم القيمة في صفة سطر موضوع الرسالة على صفحة النقاش preload - لو ذكرت، ستستخدم الصفحة المذكورة في القيمة في التحميل المسبق لمحتوى الرسالة على صفحة النقاش preloadparams[] – لو ذكرت، ستستخدم القيمة لاستبدال المعطيات الموجودة في قالب التحميل المسبق؛ كل مع | 4 | 2 | 2 | miss | 5 | 4 | 3 | miss |
Wie setze ich mein Wiki-Passwort mit meinem Benutzernamen oder meiner E-Mail-Adresse zurück?Expected passage## Muss ich meine E-Mail-Adresse bestätigen, um das Zurücksetzen zu ermöglichen? Nein, du musst deine E-Mail-Adresse nicht bestätigen, um Passwort-Rücksetzungsmails zu erzeugen. Nichtsdestotrotz wird dringend empfohlen, deine E-Mail-Adresse zu bestätigen. Dadurch kannst du sicher | 1 | 3 | 3 | 1 | 2 | 5 | 2 | miss |
What are the main navigation areas available on a MediaWiki page?Expected passageThe sidebar gives you access to important pages in the wiki such as recent changes or upload file. MediaWiki requires you to log in before seeing all of the sidebar options. Links (often called tabs) which relate to the page currently displayed: its associated discussion page, th | 3 | miss | 5 | 3 | 1 | miss | miss | 2 |
¿Durante cuánto tiempo se conserva la dirección IP asociada a una edición hecha con una cuenta temporal?Expected passageRestringimos el acceso a los datos de direcciones IP de las cuentas temporales a los usuarios que requieren de este acceso con fines de moderación antiabuso. Los criterios de acceso a estos datos se articulan en una política legal llamada Acceso a las direcciones IP de las cuenta | 3 | 1 | 1 | 1 | 1 | 5 | 2 | 4 |
ما المعلومات التي تتلقاها خدمات الترجمة الخارجية، وما القيد المتعلق بالبيانات الشخصية عند ربطها؟Expected passage### توسيع الدعم اللغوي بخدمات الترجمة الجديدة تم تصميم ترجمة المحتوى كمنصة قابلة للتوسيع. لذلك من الممكن تطوير عملاء جدد لدمج خدمات الترجمة الإضافية. بعض الاعتبارات حول طريقة تكامل خدمات الترجمة: يتم نشر الترجمات الآلية وتصحيحات المستخدم التي تم إجراؤها كجزء من بيانات الترجمات ال | 1 | 2 | 3 | 4 | 4 | 3 | miss | miss |
Notes
- The corpus is 33 help pages from mediawiki.org (Help namespace, public domain under CC0), each taken in English, French, Spanish, German and Arabic: 165 documents, fetched as plain text on 14 November 2026. Pages were picked at random among the 106 help pages translated into all five languages, keeping those between 2,500 and 40,000 characters whose Arabic version is mostly in Arabic.
- The 100 questions cover the five languages: 33 in English, 21 in French, 16 in German, 15 in Spanish and 15 in Arabic. The generator wrote every question in English, so during review we translated 67 of them into the language of the passage they were written from.
- Each question expects the passage in its own language. The same passage exists in the four other languages: a model that returns one of those instead gets no credit for it, even though it answers the question. Scores here measure same-language retrieval inside a mixed-language corpus, and are lower than they would be on five separate corpora.
- BGE Base EN v1.5 is English-only: it is included as a baseline to show what a monolingual model loses. It reads at most 380 tokens per chunk in Truvec and Cohere Embed Multilingual v3 at most 512, so both saw some of the longer chunks cut short.
- With 100 questions, a gap of a couple of points between two models is within the noise, and each language has only 15 to 33 questions: per-language differences need more questions than this.
Methodology
Truvec split the documents into chunks, generated a golden set of questions with gpt-5.6-luna (each question is linked to the chunks that answer it), embedded the corpus and the questions with every model, and retrieved the top 5 chunks for each question by cosine similarity. Every model saw the same chunks and the same questions. Scores are averaged over all 100 questions.
- Hit rate
- Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
- Recall
- Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
- MRR
- Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
- nDCG
- Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
- Precision
- Share of the top k results that are relevant. Low precision means more noise sent to the LLM.
Downloads: full report (PDF) - benchmark.json - benchmark.csv - this post as Markdown
OpenAI text-embedding-3-large won here, on these documents. Find the model that wins on yours: run the same benchmark on your documents with Truvec.