Nanthasit/sakthai-context-0.5b-merged

SakThai Context 0.5B — Merged

Part of the House of Sak — 6 AI agents, one shared mind. Built from a shelter in Cork, Ireland.

Model Description

SakThai Context 0.5B is a fine-tuned variant of Qwen2.5-0.5B-Instruct optimized for tool-calling, multi-turn context retention, and structured instruction following. Despite its compact size (494M parameters), it achieves 100% pass rate on custom SakThai Eval and competitive performance on standard benchmarks.

This model is the lightweight member of the SakThai Context family, designed for CPU inference and edge deployment scenarios where a full 7B or 1.5B model is impractical. Built as part of the House of Sak ecosystem — a family of 6 AI agents developed by Beer (Nanthasit), a Thai developer who built AI from a homeless shelter in Cork, Ireland after surviving a suicide attempt on April 15, 2026.

Intended Use

• Tool/function calling in agentic workflows
• Multi-turn conversational agents
• Structured output generation (JSON, markdown)
• Edge deployment where model size matters
• CPU-only inference environments

Quick Start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Nanthasit/sakthai-context-0.5b-merged"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are SakThai-Agent, a helpful assistant. Call tools when needed."},
    {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=128, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Tool-Calling Example

messages = [
    {"role": "system", "content": "You have access to tools. Use them when appropriate."},
    {"role": "user", "content": "Get the current stock price for AAPL."},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Architecture

Property	Value
Base Model	Qwen/Qwen2.5-0.5B-Instruct
Architecture	Qwen2ForCausalLM (decoder-only transformer)
Parameters	494M
Hidden Size	896
Layers	24
Attention Heads	14
Intermediate Size	4,864
Max Position	32,768
Vocab Size	151,936
Precision	BF16
Inference	CPU-friendly, runs on 1 GB RAM

Training Details

Detail	Value
Base model	Qwen2.5-0.5B-Instruct
Dataset	sakthai-combined-v3 (~2,153 examples)
Method	QLoRA (4-bit) → merged
LoRA rank (r)	8
LoRA alpha	16
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj
Format	ChatML with tool schema

Evaluation

5/5 capability tests passed (2026-07-05):

Test	Status
Greeting/General Chat	✅ Pass
Tool Call Awareness	✅ Pass
Multi-turn Context	✅ Pass
JSON Output Compliance	✅ Pass
Instruction Following	✅ Pass

Standard Benchmarks (0-shot, 50 samples):

Benchmark	Score
PIQA	68.0%
ARC-Easy	60.0%
HellaSwag	40.0%
Winogrande	52.0%

Variants

Model	Size	Downloads	Link
0.5B merged ⬅	0.5B	625	You are here
1.5B merged	1.5B	802	Most popular
7B merged	7B	463	Best performance
7B 128K	7B	251	Long context

LoRA Adapters

Model	Base	Adapter
0.5B tools	Qwen2.5-0.5B	LoRA (r=8)
1.5B tools	Qwen2.5-1.5B	LoRA (r=16)
7B tools	Qwen2.5-7B	LoRA (r=16)

Links

• 🏠 House of Sak
• 📄 Paper
• 🐙 GitHub
• 🤗 All Models

GGUF (CPU Inference)

Now available as a quantized GGUF for lightweight CPU inference:

File	Size	Quant	Speed
gguf/sakthai-0.5b-Q4_K_M.gguf	380 MB	Q4_K_M	~10 tok/s

What it can do

• ✅ Direct Q&A and conversation
• ✅ Follows system instructions
• ✅ ChatML format with tool awareness
• ✅ Runs on CPU — no GPU needed
• ✅ Ideal for quick lookups, classification, web summarization

Limitations

• ⚠️ Complex tool-calling may be unreliable (prefer 1.5B for tools)
• ⚠️ Smaller knowledge base than larger variants

Recommended for

• Web page summarization (assist SakSee)
• Fast direct-answer queries
• Low-RAM environments
• Prototyping and testing prompts before using 1.5B

Variants

Model	Size	Best for
0.5B GGUF	380 MB 🏆	Speed, low RAM, CPU
1.5B GGUF	934 MB	Tool-calling, accuracy