Feature Extraction
Transformers
PyTorch
Urdu
bert
BERT
encoder
embeddings
TiME
size:m
text-embeddings-inference
Instructions to use dschulmeist/TiME-ur-m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dschulmeist/TiME-ur-m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="dschulmeist/TiME-ur-m")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("dschulmeist/TiME-ur-m") model = AutoModel.from_pretrained("dschulmeist/TiME-ur-m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from dschulmeist/TiME-ur-m: direct link, hf CLI and curl.
- Browser
- Download file 1.11 kB
-
https://huggingface.co/dschulmeist/TiME-ur-m/resolve/main/README.md
- Command line
-
hf download hf://dschulmeist/TiME-ur-m/README.md
-
curl -L -o README.md https://huggingface.co/dschulmeist/TiME-ur-m/resolve/main/README.md
1.11 kB
| language: | |
| - ur | |
| library_name: transformers | |
| pipeline_tag: feature-extraction | |
| tags: | |
| - BERT | |
| - encoder | |
| - embeddings | |
| - TiME | |
| - ur | |
| - size:m | |
| license: apache-2.0 | |
| teacher_model: FacebookAI/xlm-roberta-large | |
| datasets: | |
| - uonlp/CulturaX | |
| # TiME Urdu (ur, m) | |
| Monolingual BERT-style encoder that outputs embeddings for Urdu. | |
| Distilled from FacebookAI/xlm-roberta-large. | |
| ## Specs | |
| - language: Urdu (ur) | |
| - size: m | |
| - architecture: BERT encoder | |
| - layers: 6 | |
| - hidden size: 768 | |
| - intermediate size: 3072 | |
| ## Usage (mean pooled embeddings) | |
| ```python | |
| from transformers import AutoTokenizer, AutoModel | |
| import torch | |
| repo = "dschulmeist/TiME-ur-m" | |
| tok = AutoTokenizer.from_pretrained(repo) | |
| mdl = AutoModel.from_pretrained(repo) | |
| def mean_pool(last_hidden_state, attention_mask): | |
| mask = attention_mask.unsqueeze(-1).type_as(last_hidden_state) | |
| return (last_hidden_state * mask).sum(1) / mask.sum(1).clamp(min=1e-9) | |
| inputs = tok(["example sentence"], padding=True, truncation=True, return_tensors="pt") | |
| outputs = mdl(**inputs) | |
| emb = mean_pool(outputs.last_hidden_state, inputs['attention_mask']) | |
| print(emb.shape) | |
| ``` | |