Instructions to use vultr/VultronRetrieverPrime-Qwen3.5-8B-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use vultr/VultronRetrieverPrime-Qwen3.5-8B-v2 with sentence-transformers:
from sentence_transformers import MultiVectorEncoder model = MultiVectorEncoder("vultr/VultronRetrieverPrime-Qwen3.5-8B-v2") queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - Notebooks
- Google Colab
- Kaggle
VultronRetrieverPrime-Qwen3.5-8B-v2
VultronRetrieverPrime v2 is the second release of the flagship of the VultronRetriever family, a late-interaction (ColBERT-style) retriever that scores document pages directly from their rendered image: layout, tables, charts and text, across six languages. v2 moves from 320 to 4096-dimensional multi-vectors and runs its full-attention layers bidirectionally.
The family has three tiers: the 8B Prime (this model) for maximum accuracy, the 4.5B Core for the accuracy/footprint mid-point, and the 0.8B Flash for latency- and footprint-sensitive serving. Trained and evaluated on Vultr Cloud.
Highlights
- ViDoRe V3: 67.46 mean nDCG@10 over the 8 public tasks.
- MTEB: V1 93.35, V2 72.91, V3 67.46.
- 8.4B parameters, 4096-dim, ~17 GB bf16, runs on one GPU.
- Smaller vectors: V3 holds at 67.17 with 1024 dims and 66.40 with 256.
- Six languages (en, fr, de, es, it, pt).
Results
Scores are nDCG ร100 from mteb 2.26.3, using 4096-dim embeddings and up to 1,792 visual tokens per page.
ViDoRe V3 (nDCG@10)
| Industrial | Finance EN | Finance FR | Pharma | Energy | Physics | Computer Science | HR | Mean |
|---|---|---|---|---|---|---|---|---|
| 59.65 | 72.62 | 57.04 | 70.96 | 73.03 | 53.54 | 82.84 | 70.00 | 67.46 |
ViDoRe V1 (nDCG@5)
| ArxivQA | DocVQA | InfoVQA | ShiftProject | SynAI | SynEnergy | SynGov | SynHealth | TabFQuAD | TAT-DQA | Mean |
|---|---|---|---|---|---|---|---|---|---|---|
| 93.53 | 70.37 | 95.36 | 95.55 | 99.63 | 97.63 | 99.26 | 100.00 | 97.74 | 84.46 | 93.35 |
ViDoRe V2 (nDCG@5)
| BioMedical Lectures | ESG Reports HL | ESG Reports | Economics Reports | Mean |
|---|---|---|---|---|
| 70.98 | 79.24 | 73.08 | 68.35 | 72.91 |
Per-task mteb result files are in results/. The 2 private V3 tasks are scored by the ViDoRe maintainers.
Smaller embeddings
The token vectors can be shortened: keep the first N dimensions of each vector and re-normalize it. The index shrinks in proportion. Scores are nDCG ร100 on the full benchmarks at each size.
| Dim | Index size | V1 | V2 | V3 (8 public) |
|---|---|---|---|---|
| 4096 | 1ร | 93.35 | 72.91 | 67.46 |
| 2048 | 1/2 | 93.28 | 72.85 | 67.37 |
| 1024 | 1/4 | 93.29 | 73.06 | 67.17 |
| 512 | 1/8 | 93.14 | 73.13 | 66.79 |
| 256 | 1/16 | 92.73 | 71.80 | 66.40 |
ViDoRe V3 per task (nDCG@10):
| Dim | Industrial | Finance EN | Finance FR | Pharma | Energy | Physics | Computer Science | HR | Mean |
|---|---|---|---|---|---|---|---|---|---|
| 4096 | 59.65 | 72.62 | 57.04 | 70.96 | 73.03 | 53.54 | 82.84 | 70.00 | 67.46 |
| 2048 | 59.68 | 72.47 | 57.07 | 70.86 | 72.98 | 53.41 | 82.64 | 69.90 | 67.37 |
| 1024 | 59.15 | 72.43 | 56.99 | 70.57 | 72.64 | 53.33 | 82.72 | 69.54 | 67.17 |
| 512 | 58.63 | 72.06 | 57.02 | 70.08 | 72.28 | 52.92 | 82.52 | 68.77 | 66.79 |
| 256 | 58.23 | 71.41 | 56.87 | 69.22 | 72.22 | 52.48 | 82.21 | 68.56 | 66.40 |
ViDoRe leaderboard
Top 10 by mean nDCG@10 over the 8 public ViDoRe V3 tasks; V1, V2 and the 10-task V3 mean are shown alongside.
| Model | Params | Dim | V1 | V2 | V3 (8 public) | V3 (all 10) |
|---|---|---|---|---|---|---|
| VultronRetrieverPrime-Qwen3.5-8B-v2 (this model) | 8.4B | 4096 | 93.35 | 72.91 | 67.46 | pending |
| athrael-soju/ColQwen3.5-4.5B-v4 | 4.5B | 2048 | 93.09 | 69.78 | 67.14 | pending |
| tencent/EVIE-8B | 8.4B | 4096 | 92.90 | 74.12 | 66.45 | 66.24 |
| Microsoft/microsoft-embedding-1-large | โ | 1024 | โ | โ | 66.24 | โ |
| tencent/EVIE-4.5B | 4.5B | 2048 | 92.86 | 72.92 | 65.88 | 65.70 |
| webAI-Official/webAI-ColVec1.1-8b | 8.4B | 640 | 91.53 | 67.08 | 65.32 | 64.95 |
| perplexity-ai/pplx-embed-v2-late-9b | 8.4B | 128 | 90.67 | 68.96 | 64.95 | 64.77 |
| VultronRetrieverPrime-Qwen3.5-8B (v1) | 8.4B | 320 | 92.08 | 68.18 | 64.72 | 64.26 |
| webAI-Official/webAI-ColVec1-9b | 9.4B | 2560 | 91.30 | 65.82 | 64.45 | 63.00 |
| webAI-Official/webAI-ColVec1.1-4b | 4.5B | 640 | 90.91 | 66.60 | 64.24 | 63.90 |
V1/V2: mean nDCG@5. V3 (8 public): mean nDCG@10 over the 8 public ViDoRe V3 tasks. V3 (all 10) adds the 2 private tasks (Nuclear, Telecom), scored by the ViDoRe maintainers on held-out corpora. Other models' figures: MTEB results repository, 2026-11-18; "โ" = not reported on that benchmark.
Per-task nDCG@10 (ViDoRe V3)
* = best on that task among the models listed. Mean is over the 8 public tasks; Nuc and Tel are the private tasks.
| Model | CS | Energy | FinEn | FinFr | Hr | Ind | Nuc | Pharma | Phys | Tel | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|
| VultronRetrieverPrime-Qwen3.5-8B-v2 (this model) | 82.8 | 73.0* | 72.6* | 57.0* | 70.0 | 59.6* | pending | 71.0 | 53.5* | pending | 67.46 |
| athrael-soju/ColQwen3.5-4.5B-v4 | 82.9* | 72.1 | 72.5 | 56.1 | 70.2* | 59.3 | pending | 70.8 | 53.3 | pending | 67.14 |
| tencent/EVIE-8B | 82.1 | 72.2 | 71.2 | 55.8 | 68.9 | 59.1 | 56.6* | 70.3 | 52.0 | 74.3* | 66.45 |
| Microsoft/microsoft-embedding-1-large | 82.6 | 72.5 | 69.4 | 54.5 | 68.5 | 59.3 | โ | 71.2* | 51.9 | โ | 66.24 |
| tencent/EVIE-4.5B | 82.0 | 72.0 | 69.9 | 54.7 | 67.6 | 58.8 | 56.4 | 70.0 | 52.1 | 73.5 | 65.88 |
| webAI-Official/webAI-ColVec1.1-8b | 80.1 | 70.1 | 71.9 | 54.9 | 68.5 | 57.6 | 53.7 | 67.9 | 51.5 | 73.3 | 65.32 |
| perplexity-ai/pplx-embed-v2-late-9b | 79.5 | 71.7 | 71.6 | 53.9 | 67.7 | 56.2 | 54.5 | 68.9 | 50.1 | 73.6 | 64.95 |
| VultronRetrieverPrime-Qwen3.5-8B (v1) | 79.8 | 70.3 | 69.0 | 54.5 | 66.8 | 57.4 | 53.6 | 68.2 | 51.7 | 71.3 | 64.72 |
| webAI-Official/webAI-ColVec1-9b | 80.9 | 69.8 | 68.3 | 53.7 | 70.0 | 57.2 | 47.7 | 67.3 | 48.4 | 66.7 | 64.45 |
| webAI-Official/webAI-ColVec1.1-4b | 80.3 | 69.5 | 69.2 | 53.1 | 66.9 | 56.4 | 53.3 | 67.2 | 51.2 | 71.8 | 64.24 |
Intended use
- Visual document retrieval / multimodal RAG over PDFs, scans, slides and reports, including pages with layout, tables, charts and figures.
- Multilingual document collections (en, fr, de, es, it, pt).
- Maximum-accuracy retrieval with 4096-dim multi-vectors.
Out of scope: text-only semantic search, where a single-vector dense embedder is cheaper; generative QA (this is a retriever; pair it with a reader/LLM).
Method
Per-token MaxSim scoring captures fine-grained matches against tables, figures, and layout that a single-vector embedder averages away.
- Base:
Qwen/Qwen3.5-9B(hybrid GatedDeltaNet + full-attention backbone). - Late-interaction retriever (
ColQwen3_5): 4096-dim multi-vector embeddings, MaxSim scoring, image + text inputs. - Size: 8.4B parameters, the generative head dropped for retrieval (HF reports 8B).
- 4096-dim head: trained and operated at 4096 dimensions.
- Bidirectional attention: the eight full-attention layers attend in both directions; the GatedDeltaNet layers stay causal.
- Hard negatives: mined with a retriever, then filtered with a reranker that removes negatives that are actually relevant.
- Model merging: several full-weight checkpoints merged per-block into one checkpoint.
- Trained at up to 1280 visual tokens, evaluated and deployed at 1792.
Training data
An enhanced, multilingual mixture of public and synthetic visual-document retrieval sources, spanning en, es, de, fr, it and pt. The training recipe and the assembled training dataset are not distributed in this repository.
Inputs and outputs
- Input: document-page images (RGB) and/or text queries; pages encode at up to 1792 visual tokens.
- Output: multi-vector embeddings, one 4096-dim vector per token (not a single pooled vector).
- Scoring: late-interaction MaxSim between query-token and page-token vectors.
Usage
Use
transformers>=5.2.0. Older versions ignore the config'sis_causal: false, run the model with causal attention and give different embeddings, so Sentence Transformers won't load the model on them.
Sentence Transformers
This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:
pip install "sentence-transformers[image]>=6.0.0"
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("vultr/VultronRetrieverPrime-Qwen3.5-8B-v2")
queries = [
"What is the variable represented on the y-axis of the graph?",
"Total outlay is maximum in which year?",
]
images = [
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
]
query_embeddings = model.encode_query(queries)
image_embeddings = model.encode_document(images)
# Diagonal should have higher scores
scores = model.similarity(query_embeddings, image_embeddings)
print(scores)
config.json carries dim=4096, so the projection is sized correctly at load. It also carries
text_config.is_causal=false and retrieval_attention_contract="bidirectional".
vLLM
vLLM serves this model natively through its pooling runner (the ColQwen3_5 architecture), returning
the per-token multi-vectors for late-interaction scoring. It requires a vLLM build that includes the
projection-bias fix and explicit retrieval-attention metadata support. The handler reads
retrieval_attention_contract="bidirectional" from this checkpoint and runs all eight full-attention
layers bidirectionally; it must reject missing, conflicting, or unsupported contract
metadata. The server uses the stock chat/image processor, so the ColQwen3.5
prompt contract is applied client-side: wrap each page image in the instruction template, append
the query-augmentation tokens to each query, and set the visual-token budget through
mm-processor-kwargs. Prefix caching and chunked prefill remain off for the GatedDeltaNet hybrid.
pip install vllm
import torch
from PIL import Image
from vllm import LLM
MODEL = "vultr/VultronRetrieverPrime-Qwen3.5-8B-v2"
MAX_PIXELS = 1792 * 32 * 32 # max_num_visual_tokens * (patch_size 16 * merge_size 2)^2
llm = LLM(
model=MODEL,
runner="pooling",
dtype="bfloat16",
enable_prefix_caching=False,
enable_chunked_prefill=False,
mm_processor_kwargs={"min_pixels": 65536, "max_pixels": MAX_PIXELS},
)
# ColQwen3.5 processor contract, applied client-side:
IMAGE_PROMPT = ("<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>"
"Describe the image.<|im_end|><|endoftext|>")
def query_prompt(q): return q + "<|endoftext|>" * 10 # query augmentation
images = [Image.open("page_0.png"), Image.open("page_1.png")]
queries = ["What was Q3 revenue?", "Summarize the safety findings."]
doc_out = llm.encode([{"prompt": IMAGE_PROMPT, "multi_modal_data": {"image": im}}
for im in images], pooling_task="token_embed")
qry_out = llm.encode([query_prompt(q) for q in queries], pooling_task="token_embed")
def mv(o): # one [num_tokens, 4096] multi-vector per item, L2-normalized per token
t = torch.as_tensor(o.outputs.data, dtype=torch.float32)
return torch.nn.functional.normalize(t, p=2, dim=-1)
docs, qrys = [mv(o) for o in doc_out], [mv(o) for o in qry_out]
# late-interaction MaxSim: per query token take the best doc token, then sum
scores = [[(q @ d.T).max(dim=-1).values.sum().item() for d in docs] for q in qrys]
print(scores) # scores[i][j] = relevance of query i to page j
To serve over HTTP instead:
vllm serve vultr/VultronRetrieverPrime-Qwen3.5-8B-v2 \
--runner pooling \
--no-enable-prefix-caching --no-enable-chunked-prefill \
--mm-processor-kwargs '{"min_pixels": 65536, "max_pixels": 1835008}'
Apply the same image template and query augmentation in your client requests. See the upstream example
examples/pooling/score/colqwen3_5_rerank_online.py for the full online rerank flow.
Limitations
- Tuned for six languages (en, fr, de, es, it, pt); other languages are out of distribution.
- Late-interaction multi-vector indexes are larger than single-vector dense indexes: the trade for per-token layout/table/figure sensitivity. At 4096 dimensions, v2's index is about 13ร the size of v1's 320-dim index.
License
Apache 2.0, covering the contents of this repository: model weights, config, and evaluation results.
Built on Qwen/Qwen3.5-9B (Apache 2.0); the upstream license and attribution are retained. The
training recipe and the assembled training dataset are not distributed in this repository.
Citation
@misc{vultronretrieverprime2026v2,
title = {VultronRetrieverPrime-Qwen3.5-8B-v2: Late-Interaction Visual Document Retrieval},
author = {Georgiou, Athos (athrael-soju)},
year = {2026},
howpublished = {\url{https://huggingface.co/vultr/VultronRetrieverPrime-Qwen3.5-8B-v2}}
}
Trained and evaluated on Vultr Cloud.
- Downloads last month
- 39
