metadata
license: apache-2.0
language:
- ru
- en
base_model:
- raxtemur/trocr-base-ru
pipeline_tag: image-to-text
tags:
- htr
- trocr
- image-to-text
- transformers
datasets:
- CherryJam/ru-dialectological-stackmix
- CherryJam/ru-dialectological-fonts-aug
Russian Dialectic HTR using TrOCR
The TrOCR-base-ru model was fine-tuned on synthetic datasets generated using StackMix method and based on various fonts.
For more information, check out the GitHub repository.
Model description
TrOCR-base-ru was fine-tuned for Handwritten Russian Text Recognition in dialectological cards. The model was trained for a total of 20 epochs with a batch size of 16 using dual NVIDIA T4 GPUs. The fine-tuning process took approximately 28 hours.
Example Usage
# Load libraries
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
import matplotlib.pyplot as plt
from PIL import Image
# Load image
img_path = 'path/to/image'
image = Image.open(img_path).convert("RGB")
# Load model and processor
model_name = "Daniil-Domino/trocr-base-ru-dialectic-stackmix"
processor = TrOCRProcessor.from_pretrained(model_name)
model = VisionEncoderDecoderModel.from_pretrained(model_name)
# Preprocess and run inference
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
# Output result
print(generated_text)
# Display input image
plt.axis("off")
plt.imshow(image)
plt.show()
Metrics
Below are the key evaluation metrics on the validation set:
- CER: 14.87 %
- WER: 45.71 %
- Accuracy: 55.28 %