Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets
  • Docs
  • Enterprise
  • Pricing
    • Website
      • Tasks
      • HuggingChat
      • Collections
      • Languages
      • Organizations
    • Community
      • Blog
      • Posts
      • Daily Papers
      • Hardware
      • Learn
      • Discord
      • Forum
      • GitHub
    • Solutions
      • Team & Enterprise
      • Hugging Face PRO
      • Enterprise Support
      • Inference Providers
      • Inference Endpoints
      • Storage Buckets

  • Log In
  • Sign Up

Thox-ai
/
ThoxAir-16M-role

Text Generation
GGUF
Safetensors
English
thox
bitnet
ternary
b1.58
on-device
edge
ThoxAir
thoxos
meshstack
thoxair
ollama
local-first
Model card Files Files and versions
xet
Community
1

Instructions to use Thox-ai/ThoxAir-16M-role with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

  • Notebooks
  • Google Colab
  • Kaggle
  • Local Apps Settings
  • llama.cpp

    How to use Thox-ai/ThoxAir-16M-role with llama.cpp:

    Install (macOS, Linux)
    curl -LsSf https://llama.app/install.sh | sh
    # Start a local OpenAI-compatible server with a web UI:
    llama serve -hf Thox-ai/ThoxAir-16M-role:Q8_0
    # Run inference directly in the terminal:
    llama cli -hf Thox-ai/ThoxAir-16M-role:Q8_0
    Install from WinGet (Windows)
    winget install llama.cpp
    # Start a local OpenAI-compatible server with a web UI:
    llama serve -hf Thox-ai/ThoxAir-16M-role:Q8_0
    # Run inference directly in the terminal:
    llama cli -hf Thox-ai/ThoxAir-16M-role:Q8_0
    Build from source code
    git clone https://github.com/ggml-org/llama.cpp.git
    cd llama.cpp
    cmake -B build
    cmake --build build -j --target llama-app
    # Start a local OpenAI-compatible server with a web UI:
    ./build/bin/llama serve -hf Thox-ai/ThoxAir-16M-role:Q8_0
    # Run inference directly in the terminal:
    ./build/bin/llama cli -hf Thox-ai/ThoxAir-16M-role:Q8_0
    Use Docker
    docker model run hf.co/Thox-ai/ThoxAir-16M-role:Q8_0
  • Llama new
  • LM Studio
  • Jan
  • vLLM

    How to use Thox-ai/ThoxAir-16M-role with vLLM:

    Install from pip and serve model
    # Install vLLM from pip:
    pip install vllm
    # Start the vLLM server:
    vllm serve "Thox-ai/ThoxAir-16M-role"
    # Call the server using curl (OpenAI-compatible API):
    curl -X POST "http://localhost:8000/v1/completions" \
    	-H "Content-Type: application/json" \
    	--data '{
    		"model": "Thox-ai/ThoxAir-16M-role",
    		"prompt": "Once upon a time,",
    		"max_tokens": 512,
    		"temperature": 0.5
    	}'
    Use Docker
    docker model run hf.co/Thox-ai/ThoxAir-16M-role:Q8_0
  • Ollama

    How to use Thox-ai/ThoxAir-16M-role with Ollama:

    ollama run hf.co/Thox-ai/ThoxAir-16M-role:Q8_0
  • Unsloth Desktop
  • Docker Model Runner

    How to use Thox-ai/ThoxAir-16M-role with Docker Model Runner:

    docker model run hf.co/Thox-ai/ThoxAir-16M-role:Q8_0
  • Lemonade

    How to use Thox-ai/ThoxAir-16M-role with Lemonade:

    Pull the model
    # Download Lemonade from https://lemonade-server.ai/
    lemonade pull Thox-ai/ThoxAir-16M-role:Q8_0
    Run and chat with the model
    lemonade run user.ThoxAir-16M-role-Q8_0
    List all available models
    lemonade list
  • Atomic Chat
ThoxAir-16M-role
64.3 MB
Ctrl+K
Ctrl+K
  • 1 contributor
History: 18 commits
tommytracx's picture
tommytracx
Rebuild model card to TXF-MP standard (exemplar anatomy): status callout added, Files table rebuilt with verified SHA-256 from the repo tree, License before Provenance, Kickstarter banner and link lines removed
cb1481c verified 13 days ago
  • hf
    Upload folder using huggingface_hub 3 months ago
  • .gitattributes
    1.75 kB
    Track PNG images as LFS (TXF-MP hero media) 17 days ago
  • README.md
    5.21 kB
    Rebuild model card to TXF-MP standard (exemplar anatomy): status callout added, Files table rebuilt with verified SHA-256 from the repo tree, License before Provenance, Kickstarter banner and link lines removed 13 days ago
  • hero-thoxair.png
    1.58 MB
    xet
    Add hero-thoxair.png hero media (TXF-MP) 17 days ago
  • manifest.json
    39.4 kB
    Upload folder using huggingface_hub 3 months ago
  • thoxair-16m-role-Q8_0.gguf
    17 MB
    xet
    Upload folder using huggingface_hub 3 months ago
  • thoxair-16m-role-TQ1_0.gguf
    7.39 MB
    xet
    Upload folder using huggingface_hub 3 months ago
  • thoxair-16m-role.tern1
    6.15 MB
    xet
    Upload folder using huggingface_hub 3 months ago