kotoshu grammar tagger (English), GECToR-style — v0.1.0-mega6l
An 85.8 MB int8 ONNX token tagger that detects and corrects grammatical errors fully locally — no data leaves the machine. GECToR architecture (Omelianchuk et al. 2020): distilroberta encoder, 5,000-label closed vocabulary ($KEEP / $REPLACE / $APPEND / $DELETE / $TRANSFORM / $MERGE), iterative apply-and-retag decoding (4 rounds).
Official results — CoNLL-2014 test (1,312 sentences)
Scored with the official M2 scorer, noalt gold, --max_unchanged_words 2,
4-round iterative decode:
| system | Precision | Recall | F0.5 |
|---|---|---|---|
| this model | 0.615 | 0.280 | 0.496 |
| LanguageTool 6.6 (free) | 0.305 | 0.060 | 0.168 |
| GECToR (published, roberta-base) | ~0.61 | ~0.51 | ~0.56 |
2.9x LanguageTool; runs in ~6 ms/sentence on a laptop CPU through onnxruntime.
Runtime
gem install kotoshu
# (grammar model setup: `kotoshu setup :en --model` — HF fetch path)
Model directory contents: gec-tagger.int8.onnx (int8 quantized),
labels.json (id -> label map — do not rebuild), vocab.json +
merges.txt (distilroberta byte-level BPE).
Training data (provenance)
| corpus | sentences | license |
|---|---|---|
| cLang-8 (cleaned Lang-8, google pipeline) | 2,345,088 | research-only (NAIST terms) |
| W&I + LOCNESS (BEA-2019) | 68,616 | research-only |
| NUCLE | 57,131 | research-only (NUS) |
| FCE v2.1 | 28,337 | research-only |
| CoEdIT (quad share) | 19,484 | Apache-2.0 |
| c4_200m-gec (filtered 93,193) | 93,193 | research use |
| total | 2,611,849 | mixed — see limitations |
Recipe: 8 epochs, batch 16, LR 5e-5, 10% warmup + cosine decay, label vocab = top 5,000 from the training mix.
Where this model stands
State-of-the-art in its product class: fully local (no network), CPU
real-time (6 ms/sentence), 85.8 MB int8 ONNX, embeddable in Ruby/
Rust/TS. No competing grammar checker occupies this class —
LanguageTool 6.6 (free) measures F0.5 0.168 on the same official
benchmark vs this model's 0.496 (2.9x). This is NOT an absolute
leaderboard claim: T5-11B (0.72) and LLM-API systems score higher at
45 GB+/cloud-only; our metric is accuracy-per-MB, GECToR-class at
~1/30th the size. Measurement: official CoNLL-2014 M2 scorer, noalt
gold, --max_unchanged_words 2, 4-round iterative decode (2026-10-05).
Limitations
- Research use only. The training mix includes research-only
corpora (Lang-8/NAIST, NUCLE/NUS, W&I); the weights inherit those
terms. A commercial-use line (Apache-2.0, clean-corpus training) is
planned as
kotoshu/grammar-en-cc. - English, sentence-level, learner-text domain.
- Precision-heavy operating point: strong on corrections it makes (P 0.615), conservative on recall (R 0.280) — tuned for real-world false-positive avoidance rather than leaderboard recall.
Provenance
Trained 2026-10-03 (H100, Modal), 8 epochs, final loss 0.2192. Evaluated with nusnlp/m2scorer (py3 build). Campaign log: kotoshu TODO.grammar/5-comparison-matrix.md.
License
- Code (kotoshu gem): BSD-2-Clause
- Weights: CC-BY-NC-SA-4.0 (research-only, inherited from the training-data terms)