Instructions to use Nanthasit/sakthai-context-0.5b-merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-0.5b-merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-0.5b-merged")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-0.5b-merged")
model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-0.5b-merged", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use Nanthasit/sakthai-context-0.5b-merged with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Nanthasit/sakthai-context-0.5b-merged",
	filename="gguf/sakthai-0.5b-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Nanthasit/sakthai-context-0.5b-merged with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

LM Studio
Jan

vLLM

How to use Nanthasit/sakthai-context-0.5b-merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-0.5b-merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

SGLang

How to use Nanthasit/sakthai-context-0.5b-merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-0.5b-merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-0.5b-merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Nanthasit/sakthai-context-0.5b-merged with Ollama:
```
ollama run hf.co/Nanthasit/sakthai-context-0.5b-merged:Q4_K_M
```

Unsloth Studio

How to use Nanthasit/sakthai-context-0.5b-merged with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nanthasit/sakthai-context-0.5b-merged to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nanthasit/sakthai-context-0.5b-merged to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Nanthasit/sakthai-context-0.5b-merged to start chatting

How to use Nanthasit/sakthai-context-0.5b-merged with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Nanthasit/sakthai-context-0.5b-merged:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Nanthasit/sakthai-context-0.5b-merged with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Nanthasit/sakthai-context-0.5b-merged with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Nanthasit/sakthai-context-0.5b-merged:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Nanthasit/sakthai-context-0.5b-merged with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-0.5b-merged:Q4_K_M
```

Lemonade

How to use Nanthasit/sakthai-context-0.5b-merged with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Nanthasit/sakthai-context-0.5b-merged:Q4_K_M

Run and chat with the model

lemonade run user.sakthai-context-0.5b-merged-Q4_K_M

List all available models

lemonade list

SakThai Context 0.5B — Merged

Part of the House of Sak — 6 AI agents, one shared mind. Built from a shelter in Cork, Ireland.

Model Description

SakThai Context 0.5B is a fine-tuned variant of Qwen2.5-0.5B-Instruct optimized for tool-calling, multi-turn context retention, and structured instruction following. Despite its compact size (494M parameters), it achieves 100% pass rate on custom SakThai Eval and competitive performance on standard benchmarks.

This model is the lightweight member of the SakThai Context family, designed for CPU inference and edge deployment scenarios where a full 7B or 1.5B model is impractical. Built as part of the House of Sak ecosystem — a family of 6 AI agents developed by Beer (Nanthasit), a Thai developer who built AI from a homeless shelter in Cork, Ireland after surviving a suicide attempt on April 15, 2026.

Intended Use

Tool/function calling in agentic workflows
Multi-turn conversational agents
Structured output generation (JSON, markdown)
Edge deployment where model size matters
CPU-only inference environments

Quick Start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Nanthasit/sakthai-context-0.5b-merged"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are SakThai-Agent, a helpful assistant. Call tools when needed."},
    {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=128, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Tool-Calling Example

messages = [
    {"role": "system", "content": "You have access to tools. Use them when appropriate."},
    {"role": "user", "content": "Get the current stock price for AAPL."},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Architecture

Property	Value
Base Model	Qwen/Qwen2.5-0.5B-Instruct
Architecture	Qwen2ForCausalLM (decoder-only transformer)
Parameters	494M
Hidden Size	896
Layers	24
Attention Heads	14
Intermediate Size	4,864
Max Position	32,768
Vocab Size	151,936
Precision	BF16
Inference	CPU-friendly, runs on 1 GB RAM

Training Details

Detail	Value
Base model	Qwen2.5-0.5B-Instruct
Dataset	sakthai-combined-v3 (~2,153 examples)
Method	QLoRA (4-bit) → merged
LoRA rank (r)	8
LoRA alpha	16
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj
Format	ChatML with tool schema

Evaluation

5/5 capability tests passed (2026-07-05):

Test	Status
Greeting/General Chat	✅ Pass
Tool Call Awareness	✅ Pass
Multi-turn Context	✅ Pass
JSON Output Compliance	✅ Pass
Instruction Following	✅ Pass

Standard Benchmarks (0-shot, 50 samples):

Benchmark	Score
PIQA	68.0%
ARC-Easy	60.0%
HellaSwag	40.0%
Winogrande	52.0%

Variants

Model	Size	Downloads	Link
0.5B merged ⬅	0.5B	625	You are here
1.5B merged	1.5B	802	Most popular
7B merged	7B	463	Best performance
7B 128K	7B	251	Long context

LoRA Adapters

Model	Base	Adapter
0.5B tools	Qwen2.5-0.5B	LoRA (r=8)
1.5B tools	Qwen2.5-1.5B	LoRA (r=16)
7B tools	Qwen2.5-7B	LoRA (r=16)

GGUF (CPU Inference)

Now available as a quantized GGUF for lightweight CPU inference:

File	Size	Quant	Speed
`gguf/sakthai-0.5b-Q4_K_M.gguf`	380 MB	Q4_K_M	~10 tok/s

What it can do

✅ Direct Q&A and conversation
✅ Follows system instructions
✅ ChatML format with tool awareness
✅ Runs on CPU — no GPU needed
✅ Ideal for quick lookups, classification, web summarization

Limitations

⚠️ Complex tool-calling may be unreliable (prefer 1.5B for tools)
⚠️ Smaller knowledge base than larger variants

Recommended for

Web page summarization (assist SakSee)
Fast direct-answer queries
Low-RAM environments
Prototyping and testing prompts before using 1.5B

Variants

Model	Size	Best for
0.5B GGUF	380 MB 🏆	Speed, low RAM, CPU
1.5B GGUF	934 MB	Tool-calling, accuracy

SakThai Model Family

Model	Size	Type	Downloads
1.5B-merged	934 MB	Tool-calling GGUF	942
0.5B-merged	380 MB	Lightweight GGUF	785
7B-merged	15 GB	Full-size model	534
7B-128K	15 GB	Extended context	324
Coder-1.5B	1.1 GB	Code GGUF	15
Embedding	80 MB	Search	28
Vision-7B	3.9 GB	Multimodal GGUF	0
TTS-Model	141 MB	Speech GGUF	0
Multilingual Embedding	80 MB	50+ languages	0

Full collection →

Benchmark Results

Test	Result
BFCL Tool-Calling (1.5B)	4/5
BFCL Tool-Calling (0.5B)	3/5
Coding Tasks (Coder)	5/5
Speed (0.5B)	~13 tok/s
Speed (1.5B)	~3 tok/s

Benchmark Comparison vs Similar-Sized Models

Model	MMLU	BBH	Tool-Calling
SakThai 1.5B (ours)	—	—	4/5 ⭐
SakThai 0.5B (ours)	—	—	3/5 ⭐
Qwen2.5-1.5B-Instruct (base)	56.7	38.4	~2/5
Phi-2 (2.7B)	56.8	44.5	~2/5
Gemma-2B-it	40.2	33.5	~1/5
TinyLlama-1.1B-Chat	34.8	29.1	~1/5

Key finding: SakThai models excel at tool-calling — the area most 1-3B models struggle with. Fine-tuned on a curated dataset of 1,328 high-quality tool-calling examples.

MMLU/BBH scores from official papers. Tool-calling from our BFCL benchmark (CPU, llama.cpp Q4_K_M).

Ollama Deployment

Import any SakThai GGUF model into Ollama:

ollama create sakthai-1.5b -f Modelfile
ollama run sakthai-1.5b

Hardware Requirements

Model	Min RAM	Recommended	Disk
0.5B Q4_K_M	512 MB	1 GB	380 MB
1.5B Q4_K_M	1 GB	2 GB	934 MB
Coder 1.5B	1 GB	2 GB	1.1 GB
Vision 7B	4 GB	8 GB	3.9 GB
TTS 82M	256 MB	512 MB	141 MB

Training Details

Method: QLoRA (4-bit NF4)
Base model: Qwen2.5
Rank: r=16, alpha=32
Dataset: 1,328 curated tool-calling examples
Format: ChatML with JSON tool schemas
Context: 32K tokens

Downloads last month: 785

Safetensors

Model size

0.5B params

Tensor type

BF16

Model tree for Nanthasit/sakthai-context-0.5b-merged

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Adapter

(705)

this model

Dataset used to train Nanthasit/sakthai-context-0.5b-merged

Space using Nanthasit/sakthai-context-0.5b-merged 1

Collections including Nanthasit/sakthai-context-0.5b-merged

SakThai Context

Collection

Fine-tuned tool-calling models from the SakThai family — Qwen2.5-based LoRA adapters and merged checkpoints trained on curated SFT datasets. • 7 items • Updated 18 days ago

SakThai Model Family

Collection

All SakThai models and datasets — tool-calling LLMs, embeddings, code, vision, TTS, and training data. • 14 items • Updated about 7 hours ago

Evaluation results

Overall (15/15) on SakThai Eval Suite (15 tests)
self-reported

100.000
PIQA (0-shot, 50 samples) on SakThai Eval Suite (15 tests)
self-reported

68.000
GGUF Tool-Calling (2/3) on SakThai Eval Suite (15 tests)
self-reported

0.660
ARC-Easy (0-shot, 50 samples) on SakThai Eval Suite (15 tests)
self-reported

60.000
HellaSwag (0-shot, 50 samples) on SakThai Eval Suite (15 tests)
self-reported

40.000
Winogrande (0-shot, 50 samples) on SakThai Eval Suite (15 tests)
self-reported

52.000

Nanthasit
/

sakthai-context-0.5b-merged

SakThai Context 0.5B — Merged

Model Description

Intended Use

Quick Start

Tool-Calling Example

Architecture

Training Details

Evaluation

Variants

LoRA Adapters

Links

GGUF (CPU Inference)

What it can do

Limitations

Recommended for

Variants

SakThai Model Family

Benchmark Results

Benchmark Comparison vs Similar-Sized Models

Ollama Deployment

Hardware Requirements

Training Details

Model tree for Nanthasit/sakthai-context-0.5b-merged

Dataset used to train Nanthasit/sakthai-context-0.5b-merged

Space using Nanthasit/sakthai-context-0.5b-merged 1

Collections including Nanthasit/sakthai-context-0.5b-merged

SakThai Context

SakThai Model Family

Evaluation results