--- license: apache-2.0 language: - en library_name: transformers base_model: google/gemma-4-E2B-it base_model_relation: finetune pipeline_tag: text-classification tags: - numo - gemma4 - custom-code - safetensors - scientific-literature - study-design - decision-model - calibrated - local-inference ---

Numo-1-E2B by Mycel: calibrated scientific-paper decisions on hardware you own

**[Quickstart](#quickstart) · [Results](#results) · [Limitations](#limitations) · [Full evaluation](EVALUATION.md) · [Usage guide](USAGE.md) · [Cite](#citation)** DOI: [10.57967/hf/11188](https://doi.org/10.57967/hf/11188)

# Numo-1-E2B Numo-1 reads a paper's title and abstract and answers a defined question about it, such as *what kind of study is this?*, with a probability for every possible answer. Each decision takes one model pass, about 124 ms on a consumer RTX 3060, and nothing leaves your machine. - **Accurate on its tasks.** It comes within 1–2 points of the best hosted decision models, which are much larger. - **Probabilities you can trust.** It has the lowest calibration error of every model we tested. You can therefore let it decide the papers it is sure about and send the rest to a person. - **Open and local.** Apache-2.0 weights and code. It runs offline after the download. It is a **specialist**, not a general assistant. On tasks it was not trained for, it is weak (see [limitations](#limitations)). > **Research release.** Probability scores are estimates. Keep a person in the > loop for decisions that matter. ## What it decides What it decides: table of Task ID, Question, Answers

What it decides: table of Task ID, Question, Answers

Table as text

| Task ID | Question | Answers | | --- | --- | --- | | `study_approach` | What kind of study is this paper? | Six study types, such as trial, observational, review or protocol | | `research_area` | What is its main field? | Medicine · Biology · Physics · Computer science · Other | | `biomedical_relevance` | Is a paper relevant to a seed paper? | Irrelevant · Partially · Relevant | | `scientific_field` | Which broad arXiv field? | 2–8 of eight fields | | `support_boolean` | Does the evidence fully support a hypothesis? | No · Yes | | `support_grade3` | How much does it support it? | None · Partial · Full | | `consensus4`, `consensus5` | How strongly did annotators agree it is supported? | Four or five ordered bands |

The exact wording of every question and answer is in [task-contracts.json](task-contracts.json). ## Who it is for - **Systematic-review and screening tools.** Flag trials, observational studies, reviews and protocols in thousands of abstracts, and send only the uncertain ones to a reviewer. - **Research libraries and reference managers.** Tag study type and field automatically, offline. - **Groups with confidential or unpublished work.** The paper text never leaves your machine. ## Results Every model answered the same inputs. Hosted models were queried in October 2026 and were never used to train Numo. ### Study design on PubMed Bar charts of accuracy and calibration error for eleven models on 600 PubMed papers. Numo-1-E2B: 94.3% accuracy and the lowest calibration error, 1.6%.

Bar charts of accuracy and calibration error for eleven models on 600 PubMed papers. Numo-1-E2B: 94.3% accuracy and the lowest calibration error, 1.6%.

There are 600 recent PubMed papers, 150 for each design. The correct answers are the publication types assigned by the indexers at the US National Library of Medicine (NLM), not by us. Numo-1 never saw these papers during training. **With 95% intervals:** Study design on PubMed: table of Model, Accuracy, Calibration error, Numo-1 minus model (95% interval)

Study design on PubMed: table of Model, Accuracy, Calibration error, Numo-1 minus model (95% interval)

Table as text

| Model | Accuracy | Calibration error | Numo-1 minus model (95% interval) | | --- | --- | --- | --- | | Jev 1.13 (TypeSafe) | 96.2% | 1.8% | −1.8 (−3.2 to −0.5) | | Perplexity Decider v1.2 | 96.0% | 2.1% | −1.7 (−3.0 to −0.5) | | Jev 1.14 (TypeSafe) | 95.8% | 1.8% | −1.5 (−2.8 to −0.2) | | Perplexity Decider Flash v1.2 | 95.2% | 2.6% | −0.8 (−2.7 to +1.0) | | GPT-6 Luna Decisions | 95.0% | 2.4% | −0.7 (−2.3 to +1.0) | | Clef Flash (Cloudflare) | 94.8% | 6.6% | −0.5 (−2.0 to +1.0) | | **Numo-1-E2B (local)** | **94.3%** | **1.6%** | — | | Microsoft Decision-1 | 93.5% | 4.2% | +0.8 (−1.0 to +2.7) | | Gemma 4 E2B, untrained (local) | 92.7% | 7.2% | +1.7 (−0.3 to +3.5) | | VegaML 4B (local) | 61.5% | 24.3% | +32.8 (+28.8 to +36.8) | | Laya (local) | 51.3% | 19.1% | +43.0 (+39.0 to +47.0) |

Numo-1's accuracy by design: trials 88.0%, observational 96.0%, reviews 95.3%, protocols 98.0%. ### Study approach and research area across four fields > **What are Tests C and D?** They are two test sets we built to measure > Numo-1 on papers it has never seen. > > - **The papers.** Each set has 1,600 papers. There are 400 medical papers > from Europe PMC, and 400 each from biology, physics and computer science, > drawn at random from arXiv by primary category. No paper appears in the > training data or in the other set. > - **The answers.** Two AI reviewers labelled every paper independently, > using the same written guidelines. A third AI reviewer settled the papers > where they disagreed, about 1 in 15. > - **Locked before scoring.** The answers were fixed before any model was > scored, and were read once, after Numo-1 was final. > - **One caveat.** Part of Numo's training data was labelled in the same way, > which could slightly favour Numo. The PubMed benchmark above uses > independent NLM labels and does not have this issue. > - **Not in this release.** The test sets are not included in this > repository (see [why](EVALUATION.md#2-held-out-scientific-paper-tests)). Study approach and research area across four fields: table of Model, Study approach, Test C, Study approach, Test D, Research area, Test C, Research area, Test D

Study approach and research area across four fields: table of Model, Study approach, Test C, Study approach, Test D, Research area, Test C, Research area, Test D

Table as text

| Model | Study approach, Test C | Study approach, Test D | Research area, Test C | Research area, Test D | | --- | --- | --- | --- | --- | | Jev 1.14 | 91.8% | 92.7% | 90.1% | 89.6% | | Perplexity Decider v1.2 | 92.0% | 92.7% | 90.6% | 90.1% | | Jev 1.13 | 91.2% | 92.0% | 89.1% | 89.8% | | **Numo-1-E2B (local)** | **91.3%** | **90.8%** | **88.9%** | **87.5%** | | GPT-6 Luna Decisions | 90.6% | 91.6% | 87.8% | 87.4% | | Perplexity Decider Flash v1.2 | 90.5% | 91.7% | 87.1% | 86.4% | | Microsoft Decision-1 | 91.0% | 91.4% | 82.6% | 82.7% | | Clef Flash | 89.9% | 90.9% | 84.6% | 86.8% | | Gemma 4 E2B, untrained (local) | 84.8% | 84.6% | 71.8% | 73.6% |

- **On Test C**, Numo-1 is statistically level with Jev 1.14 on study approach. - **On Test D**, the best hosted models lead by about 2 points. - **Numo-1 is the best-calibrated model** on both tasks and both tests. Its calibration error on Test D is 2.3% for study approach and 1.2% for research area. Jev 1.14 scores 2.5% and 2.2%. ### Let it decide when it is sure Line chart: accuracy rises from 91.0% to 98.0% for study approach when Numo-1 decides only the 80% of papers it is most confident about.

Line chart: accuracy rises from 91.0% to 98.0% for study approach when Numo-1 decides only the 80% of papers it is most confident about.

Let it decide when it is sure: table of Papers decided automatically, Study approach, Research area

Table as text

| Papers decided automatically | Study approach | Research area | | --- | --- | --- | | All | 91.0% | 88.2% | | Most confident 90% | 95.3% | 92.3% | | **Most confident 80%** | **98.0%** | **95.7%** | | Most confident 70% | 99.2% | 97.8% | | Most confident 50% | 99.9% | 99.4% |

These figures cover Tests C and D together, 3,200 papers per task. Check any threshold on a sample of your own papers before relying on it. ### Speed and footprint Speed and footprint: table of Numo-1-E2B

Speed and footprint: table of Numo-1-E2B

Table as text

| | Numo-1-E2B | | --- | --- | | Time per decision | 124 ms median, 175 ms p95 (RTX 3060 12 GB, one at a time) | | GPU memory | 4.7 GB peak; the large embedding table stays in CPU memory | | Network | Not needed after download | | For comparison | Hosted Jev 1.13: 240 ms median, including network |

More results are in [EVALUATION.md](EVALUATION.md), including biomedical relevance, two further test sets (Tests A and B), a check against each paper's source category and the original evidence tasks. Machine-readable values are in [evaluation-summary.json](evaluation-summary.json). ## Limitations - **A specialist, not a general model.** On six general science benchmarks from the public [Decision Index](https://github.com/apolinario/decision-index), Numo-1 averages 32.8. That is below its untrained Gemma base (48.0) and far below Jev 1.13 (77.9). Use it for the tasks above only. - **Input limits.** Each decision is limited to 1,024 tokens, including the question and the answers, and to 2–8 answers. Longer inputs are refused, never cut short. English only. - **How the references were made.** The references for Tests A–D and most of the training labels come from AI reviewers, not human experts. The PubMed benchmark uses NLM indexer labels. - **Small trade-offs on older tasks.** The final training stage cost a little accuracy on SciFact claim direction (86.0%, down from 87.7%) and on RELISH relevance (48.6%, down from 52.4%). - **Not a clinical or safety system.** It says what kind of study a paper is, not whether the study is good. Do not use it as the only basis for including a paper in a systematic review or for clinical decisions. ## Quickstart You need Linux, an NVIDIA GPU and Python 3.12 with the versions in [requirements.txt](requirements.txt). You also need a C compiler and the Python development headers, because PyTorch compiles a small helper on first use (see [setup](USAGE.md#setup)). ```python import torch from transformers import AutoModel, AutoTokenizer path = "./Numo-1-E2B" # your local model folder tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True, local_files_only=True) model = AutoModel.from_pretrained( path, trust_remote_code=True, local_files_only=True, dtype=torch.bfloat16 ).prepare_for_inference("cuda") result = model.predict( tokenizer, task="study_approach", text="Title: Vitamin D and fracture risk in older adults\n" "Abstract: We randomly assigned 2,000 participants to vitamin D or placebo ...", ) print(result["answer"], result["probabilities_by_id"]) ``` `trust_remote_code=True` runs the loading code that ships in this folder. After the download, everything works offline. The [usage guide](USAGE.md) covers every task, the input formats and custom questions. ## How it was built Numo-1-E2B starts from Google's Gemma 4 E2B. We added a small decision head and trained it, together with low-rank (LoRA) adapters on the backbone, in several supervised stages. The training data were public scientific datasets and AI-reviewed papers. The adapters are merged into the released weights. No hosted comparison model was ever used as a teacher or as a source of labels. See [training](TRAINING.md) and the [model reference](TECHNICAL.md). ## From Laya to Numo [Laya](https://github.com/NandhaKishorM/laya) is an open decision engine built on ModernBERT. It gives the model explicit answer options and makes typed decisions directly. Numo adapts that design to a Gemma backbone and to scientific tasks. It contains no Laya weights. See [why Numo](WHY_NUMO.md#from-laya-to-numo) and [credits](CREDITS.md). ## Release facts

Links: [`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it) · [LICENSE](https://huggingface.co/mycel-ai/Numo-1-E2B/blob/main/LICENSE) · [NOTICE](https://huggingface.co/mycel-ai/Numo-1-E2B/blob/main/NOTICE)

Table as text

| | | | --- | --- | | Model ID | `mycel-ai/Numo-1-E2B` | | Version | `numo-1-e2b-v1` | | Base model | [`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it), revision `3e22461f` | | Parameters | 4.63 B stored, of which 2.35 B are an embedding table kept on CPU | | Weights | 9.27 GB in four safetensors files | | Tested on | RTX 3060 12 GB with 64 GB system RAM | | Licence | Apache-2.0 for weights and code ([LICENSE](https://huggingface.co/mycel-ai/Numo-1-E2B/blob/main/LICENSE), [NOTICE](https://huggingface.co/mycel-ai/Numo-1-E2B/blob/main/NOTICE)) |

## People and credit

Links: [dev@mycel-ai.de](mailto:dev@mycel-ai.de) · [mycel-ai.de](https://mycel-ai.de)

Table as text

| | | | --- | --- | | Author | Alphin Tom | | Rights holder | Mycel UG (haftungsbeschränkt), Germany | | Contact | [dev@mycel-ai.de](mailto:dev@mycel-ai.de) · [mycel-ai.de](https://mycel-ai.de) |

Numo-1-E2B builds on the work of others, who keep their own rights: - Google DeepMind, for Gemma 4 E2B; - Nandha Kishor M, Convai Innovations and the Laya contributors, for the decision design; - the authors of the datasets listed in [CREDITS.md](CREDITS.md). Naming them does not imply that they endorse Numo. ## Citation If you use Numo-1-E2B in your work, please cite: ```bibtex @misc{tom2026numo1e2b, title = {Numo-1-E2B: Calibrated Scientific-Paper Decisions on Local Hardware}, author = {Tom, Alphin}, year = {2026}, publisher = {Mycel UG (haftungsbeschr{\"a}nkt)}, doi = {10.57967/hf/11188}, howpublished = {\url{https://huggingface.co/mycel-ai/Numo-1-E2B}}, note = {Model weights and code, version numo-1-e2b-v1, Apache-2.0} } ``` Or in text: > Tom, A. (2026). *Numo-1-E2B: Calibrated scientific-paper decisions on local > hardware* (Version numo-1-e2b-v1) [Computer software]. Mycel UG > (haftungsbeschränkt). https://doi.org/10.57967/hf/11188 When you describe how Numo works, please also credit the Gemma 4 base model and the Laya decision design.