yeniguno/turkish_agriculture_corpus
Viewer • Updated • 205k • 11
How to use yeniguno/bert-turkish-agriculture-mlm with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("fill-mask", model="yeniguno/bert-turkish-agriculture-mlm") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("yeniguno/bert-turkish-agriculture-mlm")
model = AutoModelForMaskedLM.from_pretrained("yeniguno/bert-turkish-agriculture-mlm", device_map="auto")This is a domain-adapted version ofdbmdz/bert-base-turkish-cased.
We continued masked-language pre-training on the open-sourceyeniguno/turkish_agriculture_corpus
to bias the model toward Turkish agricultural vocabulary and discourse while retaining its general-language abilities.
import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer
model_checkpoint = "yeniguno/bert-turkish-agriculture-mlm"
model = AutoModelForMaskedLM.from_pretrained(model_checkpoint)
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
text = "Sabah kahvaltıda babam, köyde bu hafta [MASK] hazırlığının başlayacağını söyledi."
inputs = tokenizer(text, return_tensors="pt")
token_logits = model(**inputs).logits
mask_token_index = torch.where(inputs["input_ids"] == tokenizer.mask_token_id)[1]
mask_token_logits = token_logits[0, mask_token_index, :]
# Pick the [MASK] candidates with the highest logits
top_5_tokens = torch.topk(mask_token_logits, 5, dim=1).indices[0].tolist()
for token in top_5_tokens:
print(f"'>>> {text.replace(tokenizer.mask_token, tokenizer.decode([token]))}'")
Base model
dbmdz/bert-base-turkish-cased