switchboard-router-0.5b-GGUF

The 994 MB build of Switchboard v0.1 (switchboardllm/switchboard-router-0.5b): a 0.5B router that reads a prompt and answers with one line of JSON, the cheapest model tier that can handle it. One F16 file for Ollama and llama.cpp on Windows, macOS and Linux. No GPU, account or API key needed.

Run it

ollama run hf.co/switchboardllm/switchboard-router-0.5b-GGUF "Write a haiku about rain"

{"tier":"small","domain":"writing"}

Or download the files and use the included Modelfile:

hf download switchboardllm/switchboard-router-0.5b-GGUF --local-dir switchboard
cd switchboard
ollama create switchboard -f Modelfile
ollama run switchboard "Unscramble the following words: eglar, ngareo, kobo"

With llama.cpp, as an OpenAI-compatible server:

llama-server -hf switchboardllm/switchboard-router-0.5b-GGUF --temp 0 -n 24 --port 8080

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json"   -d '{"messages":[{"role":"user","content":"What is the capital of France?"}],"temperature":0}'

What it answers

Field	Values
tier	small, mid or frontier: the cheapest class of model expected to answer the prompt well
domain	code, math, writing or general: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

Use it in front of your LLM calls

The quickest way is the switchboard-router package: pip install switchboard-router, then Router().pick_model(prompt). It calls this model through Ollama and falls back to frontier if anything goes wrong. To do it by hand:

import json, requests

TIER_MODEL = {
    "small": "anthropic/claude-haiku-5.5",
    "mid": "anthropic/claude-sonnet-5.5",
    "frontier": "anthropic/claude-opus-5.5",
}

def pick_model(prompt: str) -> str:
    r = requests.post("http://localhost:11434/api/chat", json={
        "model": "hf.co/switchboardllm/switchboard-router-0.5b-GGUF",
        "messages": [{"role": "user", "content": prompt}],
        "stream": False,
        "options": {"temperature": 0},
    })
    try:
        tier = json.loads(r.json()["message"]["content"])["tier"]
    except (ValueError, KeyError):
        tier = "frontier"  # fail safe
    return TIER_MODEL.get(tier, TIER_MODEL["frontier"])

The file

File	switchboard-router-0.5b-F16.gguf, 994 MB
Quantization	F16: all weights in 16 bits, no quantization
Parameters	494M (Qwen2.5-0.5B architecture)
Memory in use	about 1.3 GB
Speed	about 300 ms per routing decision on a desktop CPU

Results

Does this file route like the full model? Measured on this exact file with ollama run hf.co/switchboardllm/switchboard-router-0.5b-GGUF (temperature 0 from the repo's params file), against the full-precision model, on 347 held-out prompts: 300 random ones and 47 borderline ones picked to cover all three tiers (October 12, 2026).

	Result
Same tier as full precision, prompts sent in order	347 of 347
Same tier as full precision, prompts sent in random order	346 of 347
Valid JSON	347 of 347

Ollama reuses cached prompt prefixes between requests. On a prompt that sits right at the edge between two tiers, that can flip the decision, which happened once when we shuffled the order.

We tested smaller builds too. Q8_0 (531 MB) changed the decision on 4.0% of these prompts and Q4_K_M (398 MB) on 6.3%. For a router a changed decision is a wrong answer, so this repo ships the F16 file only.

How good is the routing? Full-precision Switchboard v0.1 on 10,000 held-out prompts:

	Switchboard	Random router*
Cost saved vs. always using Claude Opus 5.5	64.5%
Hard prompts sent to frontier	59.1%	26.2%
Under-routed (needed frontier, sent lower)	5.5%	10.0%
Invalid JSON	0

*Same share of frontier calls, picked at random. Quality is a proxy: each prompt has a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge, and small or mid counts as good enough when that answer scored 4 or more. Method and code are in the full model card.

Limitations

• This is a 0.5B router. It misroutes some prompts. 5.5% of held-out prompts were sent to a cheaper tier than they needed.
• Labels come from 2023 models and a GPT-4 judge, not human preferences. Mostly English.
• It reads one prompt at a time, not the whole conversation.
• domain comes from keyword labels and is only a hint.
• Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood or the Switchboard oracle network.

Training details are in the full model card.