---
base_model: unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit
library_name: peft
pipeline_tag: image-to-text
license: apache-2.0
language:
  - lt
datasets:
  - LuJa111/GMM-Sefai-Dataset
tags:
  - image-to-text
  - image-captioning
  - vision-language
  - lithuanian
  - lora
  - unsloth
  - qwen3-vl
---

# GMM-Sefai Team Project for University

LoRA adapter that fine-tunes **Qwen3-VL-8B-Instruct** (4-bit) to caption images in Lithuanian. Built to show captioning changes before vs. after fine-tuning.

- **Base model:** `unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit`
- **Dataset:** [`LuJa111/GMM-Sefai-Dataset`](https://huggingface.co/datasets/LuJa111/GMM-Sefai-Dataset)).
- **Method:** QLoRA via [Unsloth](https://github.com/unslothai/unsloth)

## Usage

```python
from unsloth import FastVisionModel
from PIL import Image

model, tok = FastVisionModel.from_pretrained("LuJa111/GMM-sefai-model", load_in_4bit=True)
FastVisionModel.for_inference(model)

INSTRUCTION = "Aprašyk šį vaizdą taisyklinga, sklandžia lietuvių kalba vienu ar dviem sakiniais."
img = Image.open("photo.jpg").convert("RGB")
msgs = [{"role": "user", "content": [{"type": "text", "text": INSTRUCTION}, {"type": "image"}]}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True)
inputs = tok(img, text, add_special_tokens=False, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=64, do_sample=False, repetition_penalty=1.3, no_repeat_ngram_size=3)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
```

Careful for greedy decoding GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA