Generic multilingual embedders are usually trained primarily on English with a heavy minority of other languages. "Multilingual" doesn't mean "equally good across languages" — and Arabic is one of the languages most often underserved.
The Arabic-specific challenges
- Heavy morphology. Verbs and nouns inflect for tense, person, number, gender, definiteness, mood. The same root surfaces as 50+ forms. Sparse retrieval over surface forms loses heavily without stemming/lemmatization.
- Diacritics. Most production Arabic text omits diacritics; some inserts them inconsistently. Normalize at ingest (
pyarabic.araby.strip_tashkeel). - Dialect vs MSA. Modern Standard Arabic (MSA) is the formal/written standard. Dialects (Egyptian, Gulf, Levantine, Maghrebi) diverge heavily — vocabulary, syntax, grammar. A model good at MSA may flop on Egyptian Arabic chat.
- Right-to-left rendering. Display only — embedders don't care about visual direction. But your UI must.
- Mixed-script content. Real corpora intersperse Arabic with English (esp. in tech/business). Cross-lingual embedders handle this; English-only ones drop the Arabic.
Models that actually work
- Cohere embed-multilingual-v3 — explicitly built for cross-lingual EN↔AR. Strongest off-the-shelf vendor option.
- BGE-M3 — open, multilingual, supports both dense + sparse + multi-vector. Strong Arabic. Best self-hosted choice.
- E5-multilingual — solid, open, slightly behind BGE-M3 on Arabic in MIRACL evals.
- Anthropic / OpenAI embeddings — usable but lag dedicated multilingual models on Arabic specifically.
Pre-processing pipeline for Arabic RAG
- Normalize. Strip diacritics, normalize alef variants (أ، إ، آ → ا), normalize ya variants (ى، ي → ي), strip tatweel (ـ).
- Sparse companion. Index with a stemmer (Farasa, ISRI stemmer, or AraBERT-tokenized BM25) so you catch morphological variants.
- Dense. Cohere embed-v3 or BGE-M3.
- Hybrid. RRF fuse sparse + dense.
Cross-lingual retrieval testing
Build an eval set with:
- Arabic query → Arabic doc
- Arabic query → English doc
- English query → Arabic doc
- Mixed-script query → both
If your model collapses on cross-script pairs, you have an architecture issue.
Dialect handling
- For dialect-heavy chat corpora, fine-tune the embedder on dialect query/doc pairs. Generic models will lag.
- AraBERT, MARBERT, CAMeL Lab models are stronger on dialect for downstream classification, but as embedders BGE-M3 with fine-tuning often wins.