# The best embedding model for Arabic: RAG on Arabic documents

> Arabic is one of the hardest languages for embedding models: rich morphology, optional diacritics, spelling variants, dialects and Arabizi. What to watch for, how to build an Arabic test set, and a benchmark on Arabic support and news content.

Published 2026-10-06, updated 2026-11-14 by Truvec team. Canonical: https://truvec.dev/blog/best-embedding-model-for-arabic/

Arabic is spoken by hundreds of millions of people, yet it's underrepresented in the data embedding models are trained on, and the gap between models is much wider than in English or French. A model that's excellent in English can be mediocre in Arabic, and English-only models are close to useless.

This post covers what makes Arabic retrieval hard, how to test it properly, and which models to compare. The last section benchmarks eight models on Arabic support articles and news.

## What makes Arabic hard

**Rich morphology.** Arabic builds words from roots and patterns, and attaches articles, prepositions, conjunctions and pronouns to the word itself. "وبكتابهم" ("and with their book") is one written word containing four units of meaning. The same root appears in many forms ("كتب", "كاتب", "مكتبة", "مكتوب"), and a model must understand they're related.

**Optional diacritics.** Short vowels are usually not written. Without them, one written form can be several words: "علم" can be read as "knowledge", "flag" or "he knew". Context decides, and the model must use it. Some documents (religious texts, teaching material) do include diacritics, so the same word appears in two forms.

**Spelling variants.** Alef has several forms (أ إ آ ا) that writers often mix up, final "ة" and "ه" are interchanged, as are "ى" and "ي". Users type casually; documents may be more careful. A good model treats these variants as the same word.

**Dialects.** Modern Standard Arabic (MSA) is used in documents, but users often ask in their dialect: Egyptian, Levantine, Gulf, Maghrebi. A Moroccan user's question may share few words with the MSA article that answers it.

**Arabizi and mixed text.** Many users write Arabic in Latin letters with digits ("kifach n3awed lmot de passe"), or mix Arabic with French or English. Few models handle Arabizi well.

**Tokenization.** Tokenizers built mainly on English split Arabic into many small pieces. The same text costs more tokens in Arabic, which raises embedding costs and makes input limits arrive sooner.

## Build an Arabic test set

- Write or generate questions **in Arabic**, not translated from English.
- Match your users: if they ask in **dialect**, include dialect questions against MSA documents. This is often where models differ most.
- Include **spelling variants** (alef and taa marbuta forms) and, if your users write it, **Arabizi**.
- If documents mix Arabic with French or English (common in the Maghreb and the Gulf), include questions in each language.

## Prepare the documents

- **Normalize consistently, or not at all.** Some pipelines strip diacritics and unify alef forms before embedding. This can help weaker models, but it must be applied the same way to documents and queries. Test both with and without it.
- **Check chunk sizes in tokens.** Since Arabic uses more tokens per word, a chunk size tuned for English may exceed a short-input model's limit.

## Which models to test

- **Multilingual API models**: Cohere Embed v4 and Embed Multilingual v3, Gemini Embedding, OpenAI text-embedding-3-large and 3-small.
- **Open-weight**: Qwen3 Embedding, trained on many languages and self-hostable.
- **An English-only baseline** (such as BGE Base EN), to measure the gap.

The benchmark below runs eight models on Arabic support articles and news in Modern Standard Arabic. If your users write in dialect, treat these results as an upper bound and test with dialect questions. For what each metric in the tables means, see [RAG evaluation metrics explained](/blog/rag-evaluation-metrics-that-matter/); for why a public leaderboard can't settle this for your documents, see [why MTEB alone isn't enough](/blog/mteb-benchmark-validate-embedding-model-before-switching/).

## Benchmark: Embedding models on Arabic help pages and news

Real benchmark generated by Truvec (benchmark #12, run 2026-11-14). Raw data: [benchmark.json](https://truvec.dev/blog/best-embedding-model-for-arabic/benchmark.json), [benchmark.csv](https://truvec.dev/blog/best-embedding-model-for-arabic/benchmark.csv).

- Corpus: Arabic Wikinews articles and Arabic Wikipedia help pages (https://ar.wikinews.org), 233 documents, 1,933 chunks
- Chunking: recursive, 1000 chars, 200 overlap
- Golden set: 100 questions generated by gpt-5.6-luna, human-reviewed
- Retrieval: top 5 by cosine similarity; models ranked by MRR, then recall@k

### Key results

- **Voyage 4 Large** ranks first, with an MRR of 0.970 and a hit rate of 99.0% in the top 5.
- It leads the runner-up, Qwen3 Embedding 8B, by 0.051 MRR.
- Hit rate ranges from 12.0% to 100.0% across the 8 models.
- Cheapest model within 2 points of the winner's hit rate: **Qwen3 Embedding 8B**, $0.09 to embed this corpus against $0.10.
- Fastest to embed a query: **OpenAI text-embedding-3-small** (4.2 ms per query, measured in batch from our servers).

### Retrieval quality

| Rank | Model | Provider | Hit rate@5 | Recall@5 | MRR | nDCG@5 | Precision@5 |
| --- | --- | --- | ---: | ---: | ---: | ---: | ---: |
| 1 | Voyage 4 Large | voyage | 99.0% | 99.0% | 0.970 | 0.975 | 19.8% |
| 2 | Qwen3 Embedding 8B | fireworks | 100.0% | 100.0% | 0.919 | 0.940 | 20.0% |
| 3 | Cohere Embed v4 | cohere | 100.0% | 100.0% | 0.910 | 0.933 | 20.0% |
| 4 | Gemini Embedding 2 | google | 98.0% | 98.0% | 0.907 | 0.926 | 19.6% |
| 5 | Cohere Embed Multilingual v3 | cohere | 96.0% | 96.0% | 0.887 | 0.905 | 19.2% |
| 6 | OpenAI text-embedding-3-large | openai | 98.0% | 98.0% | 0.882 | 0.907 | 19.6% |
| 7 | OpenAI text-embedding-3-small | openai | 92.0% | 92.0% | 0.802 | 0.832 | 18.4% |
| 8 | BGE Base EN v1.5 | fireworks | 12.0% | 12.0% | 0.087 | 0.096 | 2.4% |

### Hit rate by position

| Model | Top 1 | Top 2 | Top 3 | Top 4 | Top 5 |
| --- | ---: | ---: | ---: | ---: | ---: |
| Voyage 4 Large | 95.0% | 99.0% | 99.0% | 99.0% | 99.0% |
| Qwen3 Embedding 8B | 85.0% | 97.0% | 99.0% | 100.0% | 100.0% |
| Cohere Embed v4 | 85.0% | 93.0% | 97.0% | 99.0% | 100.0% |
| Gemini Embedding 2 | 84.0% | 96.0% | 98.0% | 98.0% | 98.0% |
| Cohere Embed Multilingual v3 | 83.0% | 93.0% | 93.0% | 94.0% | 96.0% |
| OpenAI text-embedding-3-large | 81.0% | 92.0% | 95.0% | 97.0% | 98.0% |
| OpenAI text-embedding-3-small | 71.0% | 85.0% | 91.0% | 91.0% | 92.0% |
| BGE Base EN v1.5 | 7.0% | 9.0% | 9.0% | 12.0% | 12.0% |

### Cost, speed and size

| Model | Price / 1M tokens | Embed the corpus | 1,000 queries | Query embedding | Search p95 | Vectors |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Voyage 4 Large | $0.12 | $0.10 | $0.0069 | 13 ms | 16 ms | 7.6 MB |
| Qwen3 Embedding 8B | $0.10 | $0.09 | $0.0057 | 27 ms | 23 ms | 30.2 MB |
| Cohere Embed v4 | $0.12 | $0.10 | $0.0069 | 16 ms | 13 ms | 11.3 MB |
| Gemini Embedding 2 | $0.20 | $0.17 | $0.01 | 42 ms | 38 ms | 22.7 MB |
| Cohere Embed Multilingual v3 | $0.10 | $0.08 | $0.0057 | 5.5 ms | 66 ms | 7.6 MB |
| OpenAI text-embedding-3-large | $0.13 | $0.11 | $0.0074 | 12 ms | 46 ms | 22.7 MB |
| OpenAI text-embedding-3-small | $0.02 | $0.02 | $0.0011 | 4.2 ms | 22 ms | 11.3 MB |
| BGE Base EN v1.5 | $0.0080 | $0.0049 | $0.0005 | 12 ms | 12 ms | 5.7 MB |

### Score separation

| Model | Relevant chunk | Best distractor | Gap |
| --- | ---: | ---: | ---: |
| Voyage 4 Large | 0.602 | 0.508 | 0.094 |
| Qwen3 Embedding 8B | 0.777 | 0.703 | 0.074 |
| Cohere Embed v4 | 0.577 | 0.505 | 0.072 |
| Gemini Embedding 2 | 0.803 | 0.761 | 0.042 |
| Cohere Embed Multilingual v3 | 0.738 | 0.683 | 0.056 |
| OpenAI text-embedding-3-large | 0.610 | 0.547 | 0.064 |
| OpenAI text-embedding-3-small | 0.561 | 0.511 | 0.048 |
| BGE Base EN v1.5 | 0.719 | 0.738 | 0.013 |

### Example questions

Rank of the first relevant chunk per model ("miss": not in the top 5).

| Question | Voyage 4 Large | Qwen3 Embedding 8B | Cohere Embed v4 | Gemini Embedding 2 | Cohere Embed Multilingual v3 | OpenAI text-embedding-3-large | OpenAI text-embedding-3-small | BGE Base EN v1.5 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| كيف تعمل المتغيرات داخل قوالب ويكيبيديا، وما نوع البيانات التي يمكن تغييرها من مقال إلى آخر؟ | 1 | 1 | 4 | 2 | 5 | 2 | 3 | miss |
| ما الحقول التي يمكن استخدامها للبحث في سجلات ويكيبيديا وتصفية نتائجها؟ | 1 | 3 | 5 | 2 | 1 | 1 | 1 | miss |
| ما الظروف الصحية والمعيشية التي واجهها الأسرى الفلسطينيون خلال الإضراب؟ | miss | 4 | 3 | 2 | 4 | 3 | miss | miss |
| ما الهدف من القاموس الإلكتروني الخاص بلغة الأينو والجهات المشاركة في تطويره؟ | 1 | 1 | 1 | 3 | 1 | 1 | 2 | miss |
| ما الدول الأوروبية الثماني التي طالبت إسرائيل بتعويضات عن منشآت هُدمت في المنطقة ج من الضفة الغربية؟ | 1 | 2 | 1 | 1 | 1 | 1 | 3 | miss |
| ماذا تعني رسالة "توقيع خام غير صالح" في إعدادات توقيع ويكيبيديا؟ | 1 | 1 | 1 | 1 | 5 | 1 | 3 | miss |
| ما أبرز الأوامر الملكية السعودية التي ذُكرت باعتبارها ذات فائدة اقتصادية للمواطنين؟ | 1 | 1 | 3 | 1 | 1 | 1 | 2 | miss |
| ما أنواع الأحداث التي يمكن للمستخدم تلقي إشعارات عنها عبر البريد الإلكتروني في ويكيبيديا؟ | 1 | 2 | 2 | 1 | miss | 1 | 3 | miss |

### Notes

- The corpus is 181 articles from Arabic Wikinews (ar.wikinews.org) and 52 help pages from Arabic Wikipedia (ar.wikipedia.org, Help namespace), both CC BY-SA 4.0, fetched as plain text on 14 November 2026. Pages were picked at random among those between 1,500 and 30,000 characters.
- All 100 questions are written in Modern Standard Arabic: 60 are about news articles and 40 about help pages. There are no dialect or Arabizi questions, so treat these results as an upper bound for dialect users.
- BGE Base EN v1.5 is English-only: it is included as a baseline.
- Arabic takes more tokens per character than English: these 1,000-character chunks average about 440 tokens. BGE Base EN v1.5 reads at most 380 tokens per chunk in Truvec and Cohere Embed Multilingual v3 at most 512, so both saw some chunks cut short.
- With 100 questions, a gap of a couple of points between two models is within the noise.

### Metric definitions

- **Hit rate**: Share of questions with at least one relevant chunk in the top k results. The most direct answer to "does the right passage reach the LLM?".
- **Recall**: Share of the relevant chunks found in the top k, averaged over questions. Matters when an answer spans several chunks.
- **MRR**: Mean reciprocal rank: 1 when the first relevant chunk is ranked first, 1/2 when second, 0 when missing. Rewards putting the answer at the top.
- **nDCG**: Normalized discounted cumulative gain: how close the ranking of all relevant chunks is to the ideal ranking.
- **Precision**: Share of the top k results that are relevant. Low precision means more noise sent to the LLM.
