The best multilingual embedding model for RAG

Multilingual RAG means questions and documents in several languages, sometimes different ones. What multilingual retrieval needs, the pitfalls of language bias and tokenization, how to test per language, and a benchmark on a five-language help center.

  • embeddings
  • rag
  • multilingual
  • benchmark

Download the benchmark report (PDF)

Most embedding benchmarks are in English, and most RAG tutorials assume English documents. Many real systems aren't: a help center in five languages, a European company's internal docs, a support bot that answers in the user's language. Multilingual retrieval has its own failure modes, and a model's English score says little about them.

This post explains what a multilingual RAG system needs from its embedding model and how to test it. The last section benchmarks the main multilingual models on a help center in English, French, Spanish, German and Arabic.

Three multilingual situations

Same language, several languages. Each user asks in their language, and the answer exists in that language. The model needs to be good in every language you serve, not just on average.

Cross-lingual. The user asks in French, the answer only exists in English. The model must map both languages into the same space, so that a French question lands near an English passage. Not all multilingual models do this well.

Mixed text. Real documents mix languages: English product names in French articles, code in Arabic tutorials, quotes in another language. The model must not be thrown off by the switch.

Know which situations you have before you test: the right model for one isn't always the right model for the others.

Pitfalls

Language bias. Many models prefer passages in the same language as the question, even when a passage in another language answers it better. In a mixed corpus, this means a French question retrieves French passages that are only roughly relevant, over the exact answer in English.

Uneven quality across languages. A model can be excellent in English and Spanish and weak in Arabic or Thai. Averages hide this: always look at results per language.

Tokenization cost. Tokenizers built mostly on English text split other languages into more tokens. The same text can cost noticeably more to embed in Arabic or Hindi than in English, and long texts may hit input limits sooner.

English-only models. Models such as BGE Base EN are trained on English. On other languages they return poor results that look plausible at first glance.

How to test

  1. Build the golden set per language, in proportion to your real traffic. Generated questions should be in the language users actually ask in.
  2. Add cross-lingual questions if users may ask in a language the answer isn't written in.
  3. Compare per language before you look at the overall ranking. A model that wins overall but fails in one language you serve is the wrong choice.
  4. Include an English-only model as a baseline. It shows how much multilingual support is worth on your data.

Candidates

Cohere's multilingual models (Embed v4, Embed Multilingual v3), Google's Gemini embeddings, Qwen3 Embedding and OpenAI text-embedding-3-large are the usual shortlist. Voyage's models are worth including as well. Check each provider's documentation for the languages it supports.

The benchmark below runs eight models on a five-language help center, with an English-only model as a baseline. Look at the example questions in each language: averages hide which languages each model struggles with. For what each metric in the tables means, see RAG evaluation metrics explained; for why a public leaderboard can't settle this for your documents, see why MTEB alone isn't enough.

Real benchmark - generated by Truvec

Benchmark: Multilingual embedding models on a 5-language help center

Corpus
MediaWiki help pages in English, French, Spanish, German and Arabic
Documents
165
Chunks
2,050
Chunking
recursive, 1000 chars, 200 overlap
Golden set
100 questions, human-reviewed
Models
8 compared
Top k
5
Run
2026-11-14

Download the full report (PDF) Every question, with the passages each model retrieved.

Key results

  • OpenAI text-embedding-3-large ranks first, with an MRR of 0.848 and a hit rate of 97.0% in the top 5.
  • It leads the runner-up, Cohere Embed Multilingual v3, by 0.037 MRR.
  • Hit rate ranges from 70.0% to 97.0% across the 8 models.
  • Cheapest model within 2 points of the winner's hit rate: Cohere Embed v4, $0.05 to embed this corpus against $0.06.
  • Fastest to embed a query: OpenAI text-embedding-3-small (5.7 ms per query, measured in batch from our servers).

Retrieval quality

Retrieval quality at top 5, ranked by MRR, then recall@k
ModelHit rate@5Recall@5MRRnDCG@5Precision@5
1OpenAI text-embedding-3-largeopenai - 3072 dims97.0%96.5%0.8480.87619.6%
2Cohere Embed Multilingual v3cohere - 1024 dims93.0%92.5%0.8110.83918.8%
3Cohere Embed v4cohere - 1536 dims95.0%94.5%0.7930.83119.2%
4Gemini Embedding 2google - 3072 dims93.0%92.5%0.7810.81518.6%
5Voyage 4 Largevoyage - 1024 dims95.0%95.0%0.7800.82319.2%
6Qwen3 Embedding 8Bfireworks - 4096 dims90.0%89.5%0.7490.78518.0%
7OpenAI text-embedding-3-smallopenai - 1536 dims88.0%88.0%0.7340.76818.0%
8BGE Base EN v1.5fireworks - 768 dims70.0%70.0%0.5830.61014.4%

Hit rate by position

How many questions the first results already answer. A model that finds the answer at position 1 lets you send fewer chunks to the LLM.

Hit rate by position: share of questions answered by the first j results
ModelTop 1Top 2Top 3Top 4Top 5
1OpenAI text-embedding-3-largeopenai - 3072 dims78.0%85.0%91.0%93.0%97.0%
2Cohere Embed Multilingual v3cohere - 1024 dims74.0%82.0%87.0%91.0%93.0%
3Cohere Embed v4cohere - 1536 dims70.0%81.0%88.0%90.0%95.0%
4Gemini Embedding 2google - 3072 dims68.0%83.0%86.0%91.0%93.0%
5Voyage 4 Largevoyage - 1024 dims68.0%79.0%88.0%91.0%95.0%
6Qwen3 Embedding 8Bfireworks - 4096 dims65.0%79.0%83.0%86.0%90.0%
7OpenAI text-embedding-3-smallopenai - 1536 dims64.0%76.0%83.0%84.0%88.0%
8BGE Base EN v1.5fireworks - 768 dims52.0%59.0%62.0%66.0%70.0%

Cost, speed and size

Embedding prices are the providers' list prices. Corpus cost is a one-time cost per model; the vector size is raw float32 storage, before the vector database's own overhead.

Cost, speed and size on this corpus
ModelPrice / 1M tokensEmbed the corpus1,000 queriesQuery embeddingSearch p95Vectors
1OpenAI text-embedding-3-largeopenai - 3072 dims$0.13$0.06$0.00398.1 ms54 ms24.0 MB
2Cohere Embed Multilingual v3cohere - 1024 dims$0.10$0.04$0.00307.8 ms49 ms8.0 MB
3Cohere Embed v4cohere - 1536 dims$0.12$0.05$0.003611 ms31 ms12.0 MB
4Gemini Embedding 2google - 3072 dims$0.20$0.09$0.006022 ms65 ms24.0 MB
5Voyage 4 Largevoyage - 1024 dims$0.12$0.05$0.003617 ms33 ms8.0 MB
6Qwen3 Embedding 8Bfireworks - 4096 dims$0.10$0.04$0.003021 ms56 ms32.0 MB
7OpenAI text-embedding-3-smallopenai - 1536 dims$0.02$0.0087$0.00065.7 ms52 ms12.0 MB
8BGE Base EN v1.5fireworks - 768 dims$0.0080$0.0032$0.000218 ms25 ms6.0 MB

Score separation

Average similarity of the best relevant chunk and of the best irrelevant one. A wide gap makes a similarity threshold reliable; a narrow one means right and wrong chunks look alike to the model.

Score separation: how clearly each model scores the right chunk above the best wrong one
ModelRelevant chunkBest distractorGap
1OpenAI text-embedding-3-largeopenai - 3072 dims0.6600.6210.039
2Cohere Embed Multilingual v3cohere - 1024 dims0.7300.6960.035
3Cohere Embed v4cohere - 1536 dims0.6280.5990.027
4Gemini Embedding 2google - 3072 dims0.8130.8000.012
5Voyage 4 Largevoyage - 1024 dims0.6160.6020.013
6Qwen3 Embedding 8Bfireworks - 4096 dims0.8050.7840.019
7OpenAI text-embedding-3-smallopenai - 1536 dims0.6930.6480.038
8BGE Base EN v1.5fireworks - 768 dims0.7610.7280.033

Example questions

Rank of the first relevant chunk for each model; miss means it wasn't in the top 5. These are the questions where the models disagree most.

Example questions from the golden set
QuestionOpenAI text-embedding-3-largeCohere Embed Multilingual v3Cohere Embed v4Gemini Embedding 2Voyage 4 LargeQwen3 Embedding 8BOpenAI text-embedding-3-smallBGE Base EN v1.5
¿Cómo decide WebFonts si usa una fuente de mi dispositivo, la descarga o la recupera de la caché?
Expected passage
## Uso Cuando la extensión está instalada y habilitada, tras el inicio de sesión, las preferencias de enlace en la página wiki, aparecerá un menú para seleccionar una fuente de la página. De forma predeterminada, la primera fuente de dicho menú se aplica a la wiki. Un usuario pue
1114352miss
¿Qué permiso se necesita para crear listas de envío de MassMessage, y dónde se puede crear una lista nueva?
Expected passage
Note that creating delivery lists requires the user to have the createwithcontentmodel user right (see phab:T92795). On most Wikimedia projects, only logged-in users have createwithcontentmodel rights. MassMessage includes a special page for managing delivery lists. To create a n
5miss43323miss
Quelles sont les étapes pour commencer une nouvelle traduction avec l'outil de traduction de contenu ?
Expected passage
Deuxième manière: cliquez sur "Contributions", puis sur le bouton "Traduction" en haut de la page. Troisième méthode: chercher la page à traduire sur le wiki dans la langue d'origine, et cliquez sur la langue de destination dans la liste des liens inter-langage de la barre vertic
33miss2155miss
كيف أشترك في موضوع نقاش ثم أوقف تلقي الإشعارات عنه لاحقًا؟
Expected passage
preloadtitle – لو ذكرت، ستستخدم القيمة في صفة سطر موضوع الرسالة على صفحة النقاش preload - لو ذكرت، ستستخدم الصفحة المذكورة في القيمة في التحميل المسبق لمحتوى الرسالة على صفحة النقاش preloadparams[] – لو ذكرت، ستستخدم القيمة لاستبدال المعطيات الموجودة في قالب التحميل المسبق؛ كل مع
422miss543miss
Wie setze ich mein Wiki-Passwort mit meinem Benutzernamen oder meiner E-Mail-Adresse zurück?
Expected passage
## Muss ich meine E-Mail-Adresse bestätigen, um das Zurücksetzen zu ermöglichen? Nein, du musst deine E-Mail-Adresse nicht bestätigen, um Passwort-Rücksetzungsmails zu erzeugen. Nichtsdestotrotz wird dringend empfohlen, deine E-Mail-Adresse zu bestätigen. Dadurch kannst du sicher
1331252miss
What are the main navigation areas available on a MediaWiki page?
Expected passage
The sidebar gives you access to important pages in the wiki such as recent changes or upload file. MediaWiki requires you to log in before seeing all of the sidebar options. Links (often called tabs) which relate to the page currently displayed: its associated discussion page, th
3miss531missmiss2
¿Durante cuánto tiempo se conserva la dirección IP asociada a una edición hecha con una cuenta temporal?
Expected passage
Restringimos el acceso a los datos de direcciones IP de las cuentas temporales a los usuarios que requieren de este acceso con fines de moderación antiabuso. Los criterios de acceso a estos datos se articulan en una política legal llamada Acceso a las direcciones IP de las cuenta
31111524
ما المعلومات التي تتلقاها خدمات الترجمة الخارجية، وما القيد المتعلق بالبيانات الشخصية عند ربطها؟
Expected passage
### توسيع الدعم اللغوي بخدمات الترجمة الجديدة تم تصميم ترجمة المحتوى كمنصة قابلة للتوسيع. لذلك من الممكن تطوير عملاء جدد لدمج خدمات الترجمة الإضافية. بعض الاعتبارات حول طريقة تكامل خدمات الترجمة: يتم نشر الترجمات الآلية وتصحيحات المستخدم التي تم إجراؤها كجزء من بيانات الترجمات ال
123443missmiss

Notes

  • The corpus is 33 help pages from mediawiki.org (Help namespace, public domain under CC0), each taken in English, French, Spanish, German and Arabic: 165 documents, fetched as plain text on 14 November 2026. Pages were picked at random among the 106 help pages translated into all five languages, keeping those between 2,500 and 40,000 characters whose Arabic version is mostly in Arabic.
  • The 100 questions cover the five languages: 33 in English, 21 in French, 16 in German, 15 in Spanish and 15 in Arabic. The generator wrote every question in English, so during review we translated 67 of them into the language of the passage they were written from.
  • Each question expects the passage in its own language. The same passage exists in the four other languages: a model that returns one of those instead gets no credit for it, even though it answers the question. Scores here measure same-language retrieval inside a mixed-language corpus, and are lower than they would be on five separate corpora.
  • BGE Base EN v1.5 is English-only: it is included as a baseline to show what a monolingual model loses. It reads at most 380 tokens per chunk in Truvec and Cohere Embed Multilingual v3 at most 512, so both saw some of the longer chunks cut short.
  • With 100 questions, a gap of a couple of points between two models is within the noise, and each language has only 15 to 33 questions: per-language differences need more questions than this.

Methodology

Truvec split the documents into chunks, generated a golden set of questions with gpt-5.6-luna (each question is linked to the chunks that answer it), embedded the corpus and the questions with every model, and retrieved the top 5 chunks for each question by cosine similarity. Every model saw the same chunks and the same questions. Scores are averaged over all 100 questions.

Hit rate
Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
Recall
Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
MRR
Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
nDCG
Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
Precision
Share of the top k results that are relevant. Low precision means more noise sent to the LLM.

Downloads: full report (PDF) - benchmark.json - benchmark.csv - this post as Markdown

OpenAI text-embedding-3-large won here, on these documents. Find the model that wins on yours: run the same benchmark on your documents with Truvec.