prestoai/qwen3-embedding-0.6b-arabic-ecom

Qwen3-Embedding-0.6B — Arabic E-commerce Search (fine-tuned)

A fine-tune of Qwen/Qwen3-Embedding-0.6B for dense retrieval of Arabic e-commerce products. Given an Arabic shopping query, the matching product embeds closest in vector space.

Trained with contrastive learning (InfoNCE) + LoRA on prestoai/arabic-ecom-data. The LoRA adapter has been merged into the base — this is a standalone model, no peft needed at load time.

⚠️ Early checkpoint. This is the step-500 of 1500 checkpoint (~33% of the planned run, ~2.3% of one epoch / ~16k pairs seen). It already beats the base model on every metric; a fuller run is expected to improve it further.

Results — before vs after fine-tuning

Dense retrieval on a held-out subset of ArabicEcomSearchData (2,901 queries = 10% sample; corpus = 22,559 judged items + 20,000 random distractors = 42,559 products), scored with the dataset's official evaluate.py. Base and fine-tuned models were evaluated on the identical queries and corpus.

Metric	Base	Fine-tuned	Δ	Relative
nDCG@10	0.4886	0.5442	+0.0556	+11.4%
Recall@10	0.4515	0.4970	+0.0455	+10.1%
MRR@10	0.5818	0.6395	+0.0577	+9.9%
Success@10 (HitRate)	0.7790	0.8352	+0.0562	+7.2%
Precision@10	0.2474	0.2721	+0.0247	+10.0%
nDCG@50	0.5582	0.6122	+0.0540	+9.7%
Recall@50	0.6739	0.7225	+0.0486	+7.2%

Read the numbers honestly: the subset uses fewer distractors than the full 107k-product corpus, so the absolute values run slightly optimistic versus the dataset's published full-corpus Meilisearch baseline (nDCG@10 = 0.6241). The trustworthy signal is the base-vs-fine-tuned delta — both models saw exactly the same data, and the fine-tune improves every metric by ~7–11%.

During-training validation (in-batch IR evaluator, 1,000 queries) at step 500: accuracy@1 = 0.647, accuracy@10 = 0.946, nDCG@10 = 0.800, MRR@10 = 0.752.

Usage

This is an asymmetric retrieval model: wrap queries in the instruction prompt; embed documents (products) as-is. The query prompt is stored on the model as prompt_name="query".

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("prestoai/qwen3-embedding-0.6b-arabic-ecom")

queries  = ["حذاء رياضي رجالي", "سماعات بلوتوث لاسلكية"]
products = ["حذاء رياضي للرجال نايك", "ساعة يد ذكية", "سماعة أذن بلوتوث لاسلكية"]

q = model.encode(queries,  prompt_name="query", normalize_embeddings=True)  # queries -> instruction
d = model.encode(products,                      normalize_embeddings=True)  # documents -> plain
scores = q @ d.T   # cosine similarity; argmax per row = best product

The full query instruction baked into the model is:

Instruct: Given an Arabic e-commerce search query, retrieve the product that best matches it
Query: {query}

Documents are rendered consistently as name | brand | categories | attributes | description (empty / placeholder fields dropped).

Training

• Base: Qwen/Qwen3-Embedding-0.6B (last-token pooling + L2 normalize)
• Objective: CachedMultipleNegativesRankingLoss (InfoNCE with in-batch + mined hard negatives; GradCache enables large effective batches)
• Data (two subsets, trained jointly):
  • positives → (query, matching product) — 578,690 train rows (in-batch negatives)
  • pairs_with_negatives → (query, product, hard negative) — 130,903 train rows
• PEFT: LoRA r=16, alpha=32, dropout=0.05, target_modules=all-linear (q/k/v/o/gate/up/down proj), task_type=FEATURE_EXTRACTION — 10M trainable params (1.6% of the base)
• Optim: lr 1e-4, cosine schedule, 5% warmup, batch 32, mini-batch 8, max_seq_length 128, seed 42
• Steps: 500 / 1500 (stopped early); fp32 on Apple Silicon (MPS)
• Split: train/val/test by hash of query text (98/1/1) — no query leaks across splits

Limitations

• Early checkpoint (33% of planned training).
• Tuned for Arabic product search; other languages/domains untested here.
• Reported metrics are on a benchmark subset (see the results note above).

Citation / links

• Base model: Qwen/Qwen3-Embedding-0.6B
• Dataset: prestoai/arabic-ecom-data
• Framework: sentence-transformers 5.6 · PEFT 0.19