metadata
base_model: unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit
library_name: peft
pipeline_tag: image-to-text
license: apache-2.0
language:
- lt
datasets:
- LuJa111/GMM-Sefai-Dataset
tags:
- image-to-text
- image-captioning
- vision-language
- lithuanian
- lora
- unsloth
- qwen3-vl
GMM-Sefai Team Project for University
LoRA adapter that fine-tunes Qwen3-VL-8B-Instruct (4-bit) to caption images in Lithuanian. Built to show captioning changes before vs. after fine-tuning.
- Base model:
unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit - Dataset:
LuJa111/GMM-Sefai-Dataset). - Method: QLoRA via Unsloth
Usage
from unsloth import FastVisionModel
from PIL import Image
model, tok = FastVisionModel.from_pretrained("LuJa111/GMM-sefai-model", load_in_4bit=True)
FastVisionModel.for_inference(model)
INSTRUCTION = "Aprašyk šį vaizdą taisyklinga, sklandžia lietuvių kalba vienu ar dviem sakiniais."
img = Image.open("photo.jpg").convert("RGB")
msgs = [{"role": "user", "content": [{"type": "text", "text": INSTRUCTION}, {"type": "image"}]}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True)
inputs = tok(img, text, add_special_tokens=False, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=64, do_sample=False, repetition_penalty=1.3, no_repeat_ngram_size=3)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Careful for greedy decoding GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA