OpenMed

OpenMed EmbeddingGemma2 BiomedicalEvidence 270M v2

OpenMed on Hugging Face OpenMed SDK on GitHub pip install openmed License Apache-2.0

OpenMed's open model for finding the answer to a medical question. It turns English health questions and answer passages into 768-dimensional vectors, so each question lands closest to the passages that answer it. Built for searching medical FAQ and patient-information answers. Trained and evaluated by OpenMed. Apache-2.0, 271.0M parameters, runs on CPU.

Developed by OpenMed
Task Medical question to answer retrieval (dense embeddings)
Output 768-dimensional vector of unit length; the dot product of two vectors is their cosine similarity
Parameters 271.0M (text encoder only)
Max input 512 tokens
Prefixes questions task: search result | query: , answers title: none | text:
Precision float32 or bfloat16, never float16
Language English
License Apache-2.0

About OpenMed

OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.

Quick start

pip install -U "sentence-transformers[image]"
import torch
from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "OpenMed/EmbeddingGemma2-BiomedicalEvidence-270M-v2",
    model_kwargs={"dtype": torch.float32},  # float32 or bfloat16, never float16
)

# The prefixes the model was trained and evaluated with
QUERY = "task: search result | query: "
DOCUMENT = "title: none | text: "

questions = [
    "How is a shingles outbreak treated?",
    "Who should get the shingles vaccine?",
]
answers = [
    (
        "Shingles causes a painful rash of fluid-filled blisters that "
        "usually appears as a band on one side of the body or face, often "
        "after a few days of burning or tingling."
    ),
    (
        "A shingles vaccine is recommended for adults aged 50 and older and "
        "lowers the risk of shingles and of long-lasting nerve pain after "
        "the rash."
    ),
    (
        "Antiviral medicines such as acyclovir, valacyclovir or famciclovir "
        "can shorten a shingles outbreak, especially when started within "
        "three days of the rash."
    ),
    (
        "A gout flare is treated with anti-inflammatory medicines such as "
        "NSAIDs, colchicine or corticosteroids, started as early as "
        "possible in the attack."
    ),
    (
        "Iron deficiency anemia can cause tiredness, weakness, pale skin, "
        "shortness of breath and cold hands and feet."
    ),
]

q = model.encode([QUERY + t for t in questions], normalize_embeddings=True)
a = model.encode([DOCUMENT + t for t in answers], normalize_embeddings=True)
scores = q @ a.T  # cosine similarity

for question, row in zip(questions, scores):
    print(question)
    for j in row.argsort()[::-1][:3]:
        print(f"  {row[j]:.3f}  {answers[j][:60]}...")
How is a shingles outbreak treated?
  0.756  Antiviral medicines such as acyclovir, valacyclovir or famci...
  0.709  A shingles vaccine is recommended for adults aged 50 and old...
  0.523  Shingles causes a painful rash of fluid-filled blisters that...
Who should get the shingles vaccine?
  0.792  A shingles vaccine is recommended for adults aged 50 and old...
  0.592  Antiviral medicines such as acyclovir, valacyclovir or famci...
  0.529  Shingles causes a painful rash of fluid-filled blisters that...

[image] adds Pillow and torchvision: the EmbeddingGemma 2 processor imports its image classes when it loads, even for text. The model in this repository is the text encoder only; there are no image, audio or video weights.

The settings

  • QUERY and DOCUMENT are the prefixes the model was trained and evaluated with. Put QUERY before every question and DOCUMENT before every answer passage, exactly as written.
  • The evaluation used title: none for every answer. The base model's format for a passage with a real title is title: {title} | text: {text}; titled passages were not evaluated after fine-tuning.
  • normalize_embeddings=True returns unit vectors, so q @ a.T is the cosine similarity the evaluation ranked by.
  • dtype: the evaluation ran in float32. bfloat16 also suits EmbeddingGemma 2 and halves memory. Do not use float16: EmbeddingGemma 2 returns NaN or degraded embeddings in float16.
  • Inputs longer than 512 tokens are cut (model.max_seq_length), the limit used in training and evaluation.

Large collections, GPU

  • Encode the answer collection once and store the vectors; at search time, encode only the question.
  • Pass batch_size=64 (or more) to model.encode for large collections.
  • On a GPU: SentenceTransformer("OpenMed/EmbeddingGemma2-BiomedicalEvidence-270M-v2", device="cuda", model_kwargs={"dtype": torch.bfloat16}).

Evaluation

MedQuAD, 1,966 held-out development questions: English health questions from nine MedQuAD collections of U.S. government health websites (CancerGov, GARD, Genetics Home Reference, MedlinePlus Health Topics, NIDDK, NINDS, NIHSeniorHealth, NHLBI and CDC), each linked by its source page to the answer it gives. OpenMed split these collections into 8,404 training questions, 3,301 calibration questions and these 1,966 development questions, keeping every source page, topic and near-duplicate answer group inside one split. Each question searches all 15,139 unique answers, and the answers its source pages gave count as relevant (2,234 for the 1,966 questions). The checkpoint (epoch 2) was chosen on the calibration questions, then the model, prefixes and scoring code were frozen and these questions were scored once. A pre-scoring check found no held-out question whose answer had the same model input as a training or calibration answer (0 excluded). These are development questions reserved by OpenMed, not an official benchmark split, so the numbers are not comparable to published MedQuAD results. Relevance comes from the source pages' own question and answer links, not from clinician judgments, and other valid answers may be unjudged.

Model NDCG@10 Recall@10 MRR@10 Recall@1
OpenMed EmbeddingGemma2 BiomedicalEvidence 270M v2 (this model) 91.76 95.79 91.12 82.95
Before OpenMed fine-tuning: google/embeddinggemma-2, same prefixes, 512 tokens 81.64 93.03 78.74 65.18

OpenMed's fine-tuning added +10.12 NDCG@10 points (95% CI +8.87 to +11.20, paired bootstrap over the 609 source groups, 2,000 resamples). The other metrics moved the same way: Recall@10 +2.76 (+1.90 to +3.85), MRR@10 +12.38 (+10.69 to +13.75) and Recall@1 +17.78 (+15.15 to +20.02). The base model already finds most right answers somewhere in its top ten; the fine-tuning mostly moves them to the top.

By source collection

MedQuAD collection Questions NDCG@10 before fine-tuning NDCG@10 Recall@10 change
CancerGov 340 71.26 84.66 +3.81
GARD 762 94.17 97.91 -0.59
Genetics Home Reference 358 74.28 93.43 +4.80
MedlinePlus Health Topics 116 92.22 93.63 +0.22
NIDDK 84 58.23 74.48 +8.83
NINDS 111 64.79 80.49 +10.81
NIHSeniorHealth 42 66.44 86.39 +8.41
NHLBI 75 80.96 93.08 +2.67
CDC 78 80.62 88.58 +4.49

NDCG@10 rose in all nine collections. Recall@10 rose in eight and fell 0.59 points in GARD. These are point estimates from the same run; the smaller collections are uncertain.

The OpenMed biomedical retrieval models

Two open OpenMed embedding models for biomedical search: one matches health questions to FAQ-style answers, the other matches research questions to literature passages. Each was scored on its own held-out questions, so their numbers are not comparable with each other.

Model Finds Base model Parameters Dimensions Max tokens Prefixes NDCG@10 before → after
OpenMed EmbeddingGemma2 BiomedicalEvidence 270M v2 (this model) the answer to a medical question (consumer health FAQ) google/embeddinggemma-2 271.0M 768 512 question and answer prefixes 81.64 → 91.76 (MedQuAD, 1,966 questions)
OpenMed GTE-ModernBERT Biomedical Embedding Base v1 evidence passages for a biomedical research question Alibaba-NLP/gte-modernbert-base 149.0M 768 384 none 85.10 → 86.81 (BioASQ, known-positive, 200 questions)

Examples

Real outputs of this model with the quick-start code (same model call, prefixes and scoring), on CPU. The 13 answers and 8 questions are synthetic, written for this card in the style of consumer health FAQ answers; none is MedQuAD text. Each question was written for one answer and searches all 13. The table gives the top three with cosine scores, the answer written for the question in bold, and where it ranks for this model and for the base model with the same code.

ID Answer
gout-symptoms Gout attacks usually start suddenly, often at night, with intense pain, swelling, redness and warmth in one joint, most often the big toe.
gout-flare A gout flare is treated with anti-inflammatory medicines such as NSAIDs, colchicine or corticosteroids, started as early as possible in the attack.
gout-prevent To prevent future gout attacks, doctors may prescribe a urate-lowering medicine such as allopurinol and advise limiting alcohol, red meat and sugary drinks.
iron-symptoms Iron deficiency anemia can cause tiredness, weakness, pale skin, shortness of breath and cold hands and feet.
iron-causes Common causes of iron deficiency are blood loss, such as heavy menstrual periods or bleeding in the gut, too little iron in the diet, and poor absorption of iron.
iron-tests Iron deficiency is diagnosed with blood tests: a complete blood count shows small, pale red blood cells, and a low ferritin level shows that the body's iron stores are used up.
shingles-symptoms Shingles causes a painful rash of fluid-filled blisters that usually appears as a band on one side of the body or face, often after a few days of burning or tingling.
shingles-vaccine A shingles vaccine is recommended for adults aged 50 and older and lowers the risk of shingles and of long-lasting nerve pain after the rash.
shingles-treatment Antiviral medicines such as acyclovir, valacyclovir or famciclovir can shorten a shingles outbreak, especially when started within three days of the rash.
cf-inheritance Cystic fibrosis is inherited in an autosomal recessive pattern: a child must receive a changed copy of the CFTR gene from each parent to have the condition.
cf-symptoms People with cystic fibrosis have thick, sticky mucus that clogs the lungs and pancreas, leading to frequent lung infections, a persistent cough and poor growth.
thyroid-symptoms An underactive thyroid can cause tiredness, weight gain, feeling cold, constipation and dry skin.
kidney-stones Kidney stones can form from uric acid, calcium or other minerals and may cause severe pain in the side or back when they move.
# Question Top three (cosine) Rank of the written answer: this model / base
1 My big toe suddenly got red, swollen and extremely painful overnight. What is going on? gout-symptoms 0.493; gout-flare 0.326; shingles-symptoms 0.278 1 / 1
2 How do you stop a gout attack? gout-prevent 0.859; gout-flare 0.667; gout-symptoms 0.621 2 / 2
3 Is there a medicine to keep gout from coming back? gout-prevent 0.774; gout-flare 0.617; gout-symptoms 0.425 1 / 1
4 Why would someone become low in iron? iron-causes 0.713; iron-symptoms 0.605; iron-tests 0.502 1 / 1
5 Which lab tests confirm iron deficiency anemia? iron-tests 0.765; iron-symptoms 0.607; iron-causes 0.511 1 / 1
6 Who should get the shingles vaccine? shingles-vaccine 0.792; shingles-treatment 0.592; shingles-symptoms 0.529 1 / 1
7 My grandmother has a painful band of blisters on one side of her chest. What could it be? shingles-symptoms 0.447; gout-symptoms 0.250; shingles-treatment 0.248 1 / 1
8 If both parents carry the gene, can their child get cystic fibrosis? cf-inheritance 0.617; cf-symptoms 0.547; iron-causes 0.065 1 / 1

Good to know

  • It answers what the question asks, not only what it is about. The pool has three answers each on gout, iron deficiency and shingles: example 4 ranks the causes answer above the symptoms and tests answers, example 5 the tests answer above the other two, and example 6 the vaccine answer above the treatment answer.
  • Plain patient wording works: examples 1 and 7 describe symptoms without naming the disease and still rank the gout and shingles symptom answers first, at lower scores (0.493 and 0.447).
  • It can miss: example 2, How do you stop a gout attack?, ranks the prevention answer first (0.859) and the flare-treatment answer second (0.667). Both answers are about gout attacks and the question's stop fits both; the base model makes the same mistake.
  • On these short examples this model and the base model both rank the written answer first on 7 of 8. With 13 answers the task is easy; the evaluation above searches 15,139, where the fine-tuning gain shows.
  • Scores are not probabilities and there is no fixed cutoff: the right first answers here score from 0.447 to 0.792. Compare scores within one question.

Limitations

  • Evaluated on MedQuAD only: English consumer health questions answered by U.S. government health websites. It has not been validated on clinical notes, health record search, scientific literature or other languages; the examples above are synthetic.
  • Relevance labels are the source pages' own question and answer links, not clinician judgments, and other valid answers among the 15,139 may be unjudged.
  • The development questions follow MedQuAD's mix of collections (GARD alone has 762 of the 1,966); they are not a sample of real patient questions.
  • Inputs are cut at 512 tokens. It was evaluated at the full 768 dimensions with title: none; the base model's shorter Matryoshka sizes (512, 256, 128) and titled passages were not evaluated after fine-tuning.
  • Text only: the base model's image, audio and video encoders are not included.
  • Cosine scores rank passages for one question; they are not calibrated probabilities.
  • The base model's pretraining data is not public, so whether it saw MedQuAD is unknown.
  • It ranks passages. It does not check that an answer is correct, current or safe for a person, does not give medical advice and is not a medical device.

Built on

OpenMed fine-tuned this model from google/embeddinggemma-2 by Google DeepMind (Apache-2.0), with the open-source sentence-transformers library. Training and evaluation data: MedQuAD, the medical question answering dataset by Asma Ben Abacha and Dina Demner-Fushman (BMC Bioinformatics, 2019), released under CC BY 4.0. OpenMed used the nine collections that include answer text and turned their question and answer pairs into retrieval training pairs. The three MedlinePlus subsets whose answers were removed from the public release (A.D.A.M. Medical Encyclopedia, drug information, herbs and supplements) and the TREC 2017 LiveQA medical test questions were not used. Thank you to Google DeepMind and to the MedQuAD authors for releasing their work openly; this credit does not imply their endorsement.

This model is released under Apache-2.0. No MedQuAD text is included in this repository.

Citation

@misc{panahi2025openmedner,
  title={OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets},
  author={Maziyar Panahi},
  year={2025},
  eprint={2508.01630},
  archivePrefix={arXiv}
}

@article{benabacha2019medquad,
  title={A Question-Entailment Approach to Question Answering},
  author={Ben Abacha, Asma and Demner-Fushman, Dina},
  journal={BMC Bioinformatics},
  volume={20},
  number={1},
  pages={511},
  year={2019},
  doi={10.1186/s12859-019-3119-4}
}

@misc{embeddinggemma2_2026,
  title={EmbeddingGemma 2: Unified and Efficient Multimodal Representations},
  author={Schechter Vera, Henrique and Dua, Sahil and Team, EmbeddingGemma},
  year={2026},
  eprint={2610.37364},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2610.37364}
}

OpenMed · Hugging Face · GitHub · PyPI · X

Downloads last month
23
Safetensors
Model size
0.3B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for OpenMed/EmbeddingGemma2-BiomedicalEvidence-270M-v2

Finetuned
(184)
this model

Papers for OpenMed/EmbeddingGemma2-BiomedicalEvidence-270M-v2

Evaluation results

  • NDCG@10 on MedQuAD (1,966 held-out development questions, 15,139 answers)
    self-reported
    91.760
  • Recall@10 on MedQuAD (1,966 held-out development questions, 15,139 answers)
    self-reported
    95.790
  • MRR@10 on MedQuAD (1,966 held-out development questions, 15,139 answers)
    self-reported
    91.120
  • Recall@1 on MedQuAD (1,966 held-out development questions, 15,139 answers)
    self-reported
    82.950