Instructions to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

SGLang

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8
```

Deployment Note: vLLM JIT Path Resolution -nvidia/cu13/lib/-

by MadGoatHaz - opened Jul 15

Discussion

MadGoatHaz

Jul 15

Because this checkpoint uses a dynamic activation scheme (W8A8 FP8_DYNAMIC), vLLM falls back to the Triton JIT path during the warmup phase. In isolated environments (like uv or pinned pip virtualenvs), nvrtc throws a runtime failure because libnvrtc-builtins.so.13.0 is localized inside the virtual environment's site-packages (nvidia/cu13/lib/) instead of standard system library paths. This causes tensor-parallel worker sub-processes to crash on initialization.

Adding this dynamic path discovery snippet to the vLLM execution instructions will help cluster operators so they don't have to manually hunt for their virtualenv layout:

# Auto-locate nested CUDA wheel libraries for the linker
export LD_LIBRARY_PATH=$(python -c "import site, os; print(next((os.path.join(p, 'nvidia/cu13/lib') for p in site.getsitepackages() if os.path.isdir(os.path.join(p, 'nvidia/cu13/lib'))), ''))"):$LD_LIBRARY_PATH

# Launch server
uv run vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 --tensor-parallel-size 8 ...

oplatek

BottleCapAI org Jul 15

We just released new version just 5 minutes after comment. Can you tried it?
I hope it might fix this as well. I tested on clean uv environment because we discovered this problem too.

MadGoatHaz

Jul 15

Sure thing, once the cluster is done with it's current task I will load it up and report back...

MadGoatHaz

Jul 18

Confirmed - Full hugging face pull with vllm and it loaded without issue.

Adamiros changed discussion status to closed Jul 23

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment