gokceai/qwen7b_baseline_v1

Qwen 2.5 7B — Enterprise Customer Support LoRA Adapter

QLoRA fine-tuned adapter for enterprise customer support conversations.

Model Details

• Base model: Qwen/Qwen2.5-7B-Instruct
• Method: QLoRA (NF4 4-bit quantization + LoRA adapter)
• LoRA config: r=4, alpha=8, target_modules=[q_proj, v_proj]
• Dataset: 963 training + 86 validation customer support conversations
• Trainable parameters: 1,261,568 (0.017% of base)
• Adapter size: 21 MB

Performance

Metric	Value
Final train loss	2.44
Final eval loss	1.81
Perplexity	6.11
Training time	14 min on 2x T4
Training cost	$0 (Kaggle free tier)

Capabilities

• Multi-turn context retention
• Empathetic clarification (no over-asking)
• Out-of-scope refusal with polite redirect
• Escalation handling without fake actions
• 6 customer tones supported (neutral, frustrated, angry, confused, urgent, non-native English)

Training Details

• Framework: PEFT 0.13.2, TRL 0.12.1, transformers 4.46.3
• Precision: FP16 (Turing T4 compatibility)
• Attention: SDPA
• Optimizer: paged_adamw_8bit
• LR: 2e-4, cosine schedule, 5% warmup
• Batch: effective 32 (per_device=1, grad_accum=32)
• Epochs: 1

Limitations

• Baseline model trained on small dataset (963 examples, 1 epoch only)
• No RAG integration — knowledge is only what's in fine-tune data
• Cannot perform real actions; only describes/initiates them
• English only; multilingual support planned

License

Apache 2.0 — same as base Qwen 2.5