commit-msg-qwen-lora

A LoRA adapter fine-tuned on top of Qwen2.5-Coder-1.5B-Instruct to generate concise, conventional-style Git commit messages from staged diffs.

Training Details

• Base model: Qwen2.5-Coder-1.5B-Instruct
• Method: QLoRA (LoRA rank 16) via Unsloth on Google Colab T4 GPU
• Dataset: 2,397 real (diff, commit message) pairs extracted and cleaned from personal GitHub repositories
• Train / Val / Test split: 1919 / 239 / 239
• Epochs: 3 | Batch size: 8 (effective) | Learning rate: 2e-4
• Training loss: 1.03 → 0.88

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-Coder-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "sharad31/commit-msg-qwen-lora")
tokenizer = AutoTokenizer.from_pretrained("sharad31/commit-msg-qwen-lora")

diff = """diff --git a/src/auth.ts b/src/auth.ts
+  if (!user || !pass) throw new Error('Missing credentials');
"""

messages = [
    {"role": "system", "content": "You are a precise commit message generator. Given a git diff, write a concise, conventional-style commit message."},
    {"role": "user", "content": f"Diff:\n```\n{diff}\n```"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=64, temperature=0.3)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))