vadimbelsky/bert-esi-triage-v58

BERT-ESI-Triage v58 — BiomedBERT multi-head triage classifier

TL;DR

bert-esi-triage-v58 is a fine-tuned BiomedBERT classifier for the Emergency Severity Index (ESI 1-5) triage decision. v58 is the successor to v57 (epoch 3) on the same 18-eval suite. Headline improvements vs v57 e3:

• MIMIC-IV-ED ESI 1 recall: +4.0pp ensemble (78.1% → 82.1%).
• MC-MED Stanford ESI 1 recall: +7.0pp (77.0% → 84.0%).
• MIETIC ensemble ESI 1 recall held at 96.7%.
• Lukina v3 ESI 1 recall held at 94.3%.
• Concept-density / frequent-flyer / polypharmacy / geriatric / multi-CC subgroup slices all win ESI 1 recall by +1.6 to +6.7pp.

Trade-off: cal_exact regresses by 2-6pp on MIETIC, judgment_gap, cardiac_arrest, and sepsis slices (a recall-for-safety shift).

Architecture differences vs v57

• No engineered-feature fusion. v57 trained with an 11-d MEWS / qSOFA / shock_index feature vector projected to 64-d and concatenated with CLS into esi_head Linear(832, 5). v58 drops the fusion path and uses esi_head Linear(768, 5) directly on CLS. The encoder learns vital patterns from raw text alone.
• Trainer. v58 uses train_bert_v44_bf16.py (bf16 native on GB10); v57 used train_bert_v44_v53_3.py (feature-fusion variant).
• Symptom head dim. v58 = 199, v57 = 203 (concept vocab pruning).

Inference

Load with PyTorch + transformers; CLS-pooled features feed each head:

• esi_head: Linear(768, 5) — main output.
• 19 auxiliary heads: symptom, flag, pain, arrival, gestalt, disposition, resource, vitals, etc.

The companion Space vadimbelsky/esi-triage-demo ships the full calibration stack (demographic normalizer → per-dialect temperature → confidence-aware ESI 1 bias → deterministic engine min ensemble).

License

• Model weights: MIT.
• Training data: private (MIMIC-IV-ED + Stanford MC-MED + MIETIC + Lukina v3 + curated synth).

⚠️ Not a medical device. Not for clinical use. This is a research artifact distributed for reproducibility and benchmarking. Real deployment requires institutional validation + IRB + local recalibration.

Performance — v58 epoch 3 vs v57 epoch 3

Per-slice ESI 1 recall (calibrated):

Slice	n	v57 cal_ESI1_R	v58 cal_ESI1_R	Δ
mimic_holdout	7,917	76.6%	80.2%	+3.6pp
mcmed_clean_holdout	1,000	77.0%	84.0%	+7.0pp
mietic_clean_holdout	200	90.0%	86.7%	-3.3pp
lukina_v3	201	94.3%	94.3%	0pp
sepsis_eval_v1	93	95.0%	95.0%	0pp
stroke_eval_v1	97	95.0%	92.5%	-2.5pp
anaphylaxis_eval_v1	90	97.1%	97.1%	0pp
cardiac_arrest_eval_v1	103	93.3%	90.0%	-3.3pp
ob_emergency_eval_v1	98	95.0%	90.0%	-5.0pp
pediatric_n200_v2	200	95.0%	95.0%	0pp
judgment_gap_v1	177	88.9%	83.3%	-5.6pp
geriatric_n200_v1	200	78.8%	80.0%	+1.3pp
polypharmacy_n200_v1	200	72.5%	76.2%	+3.7pp
frequent_flyer_n200_v1	135	80.0%	86.7%	+6.7pp
vital_completeness	231	83.3%	86.7%	+3.3pp
multi_cc_complexity	304	76.2%	77.8%	+1.6pp
concept_density	260	80.0%	86.7%	+6.7pp

Verdict: 8 wins / 5 losses / 5 neutral on ESI 1 recall (mean +0.78pp).

MIMIC per-class recall (calibrated):

Class	n	v57 R	v58 R	Δ
ESI 1	475	76.6%	80.2%	+3.6pp
ESI 2	2,582	64.6%	62.7%	-1.9pp
ESI 3	4,286	54.4%	50.1%	-4.4pp
ESI 4	554	64.6%	67.7%	+3.1pp
ESI 5	20	40.0%	40.0%	0pp

Known gaps (carried to v59)

• SYMPTOM_LABELS index mismatch. v58 symptom_head outputs 199 logits; the demo's SYMPTOM_LABELS list (inherited from v47/v49) has 178 canonical entries plus runtime padding for the extra positions. Specific concepts that grew between v57→v58 may be mislabeled in the Space UI. Tracked for v59 (task #463).
• ESI 5 recall ≤ 40% across slices — sparse class, dialect-noisy labels. v58 holds the v57 floor but does not improve it.
• MIETIC / OB / cardiac_arrest ESI 1 R regressions vs v57. Likely driven by dropping the engineered-feature signal on narrative slices where vitals are inline rather than parsed.
• Engineered features no longer in graph. Downstream stacks that computed MEWS / qSOFA / shock_index and passed them to v57 must drop that path for v58.

Companion Space

vadimbelsky/esi-triage-demo — 19-head dashboard with engine ensemble, demographic normalizer, dialect detection, per-dialect calibration. The Space loads this model and exposes the full calibrated pipeline.