LFM2.5-Embedding-350M-mxfp4 (MLX, mxfp4)

MLX mxfp4 (group_size=32, 4-bit) quantization of LiquidAI/LFM2.5-Embedding-350M, a bidirectional LFM2.5 dense bi-encoder (1024-d CLS vector) for multilingual retrieval. Runs on Apple Silicon via MLX.

  • Weights: 188 MB (mxfp4 (group_size=32, 4-bit))
  • Encode throughput: 13936 tokens/sec (M-series, this benchmark)
  • Pooling / scoring: CLS token, cosine similarity
  • Prompts: query: / document: (required โ€” trained with them)

Retrieval quality โ€” English NanoBEIR (4 datasets, 50 queries each)

Dataset NDCG@10 (fp16) NDCG@10 (mxfp4) Recall@10 (fp16) Recall@10 (mxfp4)
NanoNQ 0.7135 0.7227 0.8000 0.8400
NanoFiQA2018 0.5632 0.5831 0.6443 0.6828
NanoSciFact 0.7461 0.7512 0.8500 0.8700
NanoNFCorpus 0.3645 0.3661 0.1583 0.1565
Mean 0.5968 0.6058 0.6131 0.6373

Mean NDCG@10 retention vs fp16: 101.5% (fp16 0.5968 โ†’ mxfp4 0.6058). fp16 baseline encode: 14496 tok/s.

Benchmarked on a fixed 4-dataset English NanoBEIR subset to measure the quantization quality delta vs fp16 (not the full multilingual suite โ€” see the base model card for published numbers).

Usage (MLX)

# pip install mlx mlx-lm transformers
# This repo bundles `mlx_lfm2_encoder.py` โ€” the bidirectional LFM2 encoder
# (CLS pooling / ColBERT MaxSim) that the stock causal LFM2 loaders do NOT provide.
import mlx.core as mx
from transformers import AutoTokenizer
from mlx_lfm2_encoder import load_model

tok = AutoTokenizer.from_pretrained(".", trust_remote_code=True)
model, _ = load_model(".", head="embedding")     # head: "embedding" or "colbert"

# Asymmetric prompts (REQUIRED โ€” the model was trained with them):
def encode(texts, prefix):
    enc = tok([prefix + t for t in texts], return_tensors="np", padding=True,
              truncation=True, max_length=512)
    out = model(mx.array(enc["input_ids"]), mx.array(enc["attention_mask"]))
    mx.eval(out)
    return out  # (B, 1024) CLS-pooled, L2-normalized

q = encode(["was the nightmare before christmas a disney film"], "query: ")
d = encode(["The Nightmare Before Christmas is a 1993 stop-motion film ..."], "document: ")
scores = (q @ d.T)  # cosine similarity

Why a bundled loader?

These are bidirectional encoders (non-causal attention + non-causal short-conv + CLS pooling). General-purpose causal LFM2 loaders produce wrong embeddings here, so this repo ships mlx_lfm2_encoder.py (validated to cosine โ‰ฅ 0.999 against the original transformers model). For the broader MLX embedding ecosystem see mlx-embeddings.

License

Inherits the LFM Open License v1.0 (lfm1.0) from the base model.

Downloads last month
9
Safetensors
Model size
0.4B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for sahilchachra/LFM2.5-Embedding-350M-mxfp4

Finetuned
(12)
this model

Collection including sahilchachra/LFM2.5-Embedding-350M-mxfp4