---
language:
- bho
license: mit
tags:
- sentiment-analysis
- text-classification
- bhojpuri
- devanagari
- low-resource-nlp
- bert
datasets:
- abhiprd2000/Bhojpuri-Behavioral-Corpus-8K
metrics:
- accuracy
- f1
---

# Bhojpuri Sentiment Model (v2)

4-class sentiment classifier for native Bhojpuri text (Devanagari script),
fine-tuned from `ai4bharat/IndicBERTv2-MLM-only` on majority-vote consensus
labels from 5 open-weight LLM judges (see the
[PURVA Bhojpuri Sentiment Corpus](https://huggingface.co/datasets/abhiprd2000/Bhojpuri-Behavioral-Corpus-8K)).

## Labels

`objective`, `positive`, `negative`, `neutral` — see the dataset card for
full definitions. `objective` reports fact with no evaluative stance;
`positive`/`negative`/`neutral` apply to subjective sentences by polarity.

## Training

| | |
|---|---|
| Base model | `ai4bharat/IndicBERTv2-MLM-only` (revision `8598f13`) |
| Label source | majority vote across 5 LLM judges (not Dawid–Skene — see dataset card for why) |
| Train / dev / test | 72,168 / 9,019 / 9,020 (stratified by register × text_type × source_name, seed 42) |
| Epochs | 4 |
| Learning rate | 2e-5 |
| Batch size | 32 |
| Max sequence length | 128 |
| Precision | fp16 |
| Seed | 42 |
| Checkpoint selection | final epoch (no best-on-dev selection; dev loss used for monitoring only) |

## Results

Scored on a 108-item human-labeled gold set (`gold_test_120.jsonl` minus 3
`mixed`-labeled items, which fall outside this model's 4-class label space;
verified zero overlap with train/dev). Bootstrap 95% CI, 1,000 resamples.

| Metric | Score | 95% CI |
|---|---|---|
| Accuracy | 65.7% | [56.5%, 75.0%] |
| Macro-F1 | 50.3% | [41.3%, 59.5%] |

By register:

| Register | n | Accuracy |
|---|---:|---:|
| encyclopedic | 12 | 0.833 |
| news | 42 | 0.810 |
| literary | 19 | 0.632 |
| verse | 18 | 0.500 |
| commentary | 15 | 0.333 |
| opinion | 2 | *n<10, not reported* |

Accuracy is strong on news/encyclopedic register and weak on commentary and
verse — see the dataset card's "Known issues" for the neutral-class and
Dawid-Skene-vs-majority-vote caveats, which apply to this model too (it is
the majority-vote-trained checkpoint; a separate Dawid-Skene-trained
checkpoint scores 63.0% accuracy / 51.4% macro-F1 on the same gold set —
within this comparison's confidence interval, not a clear win either way).

## Usage

```python
from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="abhiprd2000/bhojpuri-sentiment-model",
)

texts = [
    "ई खाना बहुत स्वादिष्ट बा।",       # positive (99.8%)
    "आज बहुत थकान लागत बा।",            # negative (99.6%)
    "हम कल पटना जाइब।",                 # objective (95.7%)
]

for text in texts:
    result = classifier(text)[0]
    print(f"{text} -> {result['label']} ({result['score']*100:.1f}%)")
```

## Licence

MIT, matching the base model (`ai4bharat/IndicBERTv2-MLM-only`, confirmed
`license:mit` on its model card). Differs from v1-legacy, which was tagged
`apache-2.0`.

## Version history

- **v1-legacy** (tag `v1-legacy`): XLM-RoBERTa (`cardiffnlp/twitter-xlm-roberta-base-sentiment`)
  fine-tuned on a 68,822-row, artificially class-balanced, partly
  machine-translated dataset. Scored 97.6% accuracy / 0.976 macro-F1 on its
  own held-out 501-item balanced/translated test set.
- **v2** (this model): IndicBERTv2, trained on native non-translated text
  with naturally imbalanced labels, scored against a human-labeled gold set.
  **v1 and v2 numbers are not comparable** — different base model, different
  training data, different label source, different test set. Do not quote
  v1's 97.6% alongside v2's 65.7% as if they measure the same thing.