legalbert-primary_topic

Fine-tuned Legal-BERT classifier that assigns a primary legal topic to a multi-turn conversation.

Part of the Legal QA collection.

Model description

Stage 2 of the Legal-BERT routing pipeline. Pair with legalbert-seeks_guidance and only trust the topic label when stage 1 predicts legal guidance.

Input preprocessing: user messages only, serialized as User: content lines (assistant turns are dropped). Legal-BERT uses 512-token truncation.

Topic taxonomy

Same 14 legal topics plus (non-guidance) as modernbert-primary_topic.

Results

Split	N	Accuracy	Precision	Recall	F1
Validation (best checkpoint)	106	79.25%	78.88%	79.25%	78.42%
Test (held-out)	107	78.50%	81.45%	78.50%	78.65%

Joint pipeline test (with legalbert-seeks_guidance): legal 84.1%, topic 72.0%, joint 72.0%.

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "AmirMohseni/legalbert-primary_topic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
input_mode = getattr(model.config, "legal_cls_input_mode", "user")
max_length = getattr(model.config, "legal_cls_max_length", 512)

See legalbert-seeks_guidance for a full two-stage example.

Intended uses & limitations

Use for: topic routing baseline at 512 tokens after legal-guidance detection.

Caveats: truncation; silver labels; English only.

Training data

Dataset: AmirMohseni/WildChat-Legal-Classification-V2-Balanced

• Splits: train 1909 · val 106 · test 107
• Target field: primary_topic (empty for non-guidance rows)

Training procedure

Selected from a hyperparameter sweep. See legalbert_classifier_sweep_colab.ipynb.

Setting	Value
Base model	nlpaueb/legal-bert-base-uncased
Input mode	User turns only
Max length	512
Learning rate	1e-4
Epochs	8
Effective batch size	128 (32 × 4 grad accum)
Best checkpoint	Highest validation accuracy (79.25%)

Framework versions

• Transformers 5.9.0 · PyTorch · Datasets