Instructions to use ornith-ai/Ornith-1.0-35B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ornith-ai/Ornith-1.0-35B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ornith-ai/Ornith-1.0-35B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ornith-ai/Ornith-1.0-35B")
model = AutoModelForMultimodalLM.from_pretrained("ornith-ai/Ornith-1.0-35B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ornith-ai/Ornith-1.0-35B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ornith-ai/Ornith-1.0-35B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.0-35B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ornith-ai/Ornith-1.0-35B

SGLang

How to use ornith-ai/Ornith-1.0-35B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ornith-ai/Ornith-1.0-35B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.0-35B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ornith-ai/Ornith-1.0-35B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.0-35B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ornith-ai/Ornith-1.0-35B with Docker Model Runner:
```
docker model run hf.co/ornith-ai/Ornith-1.0-35B
```

Does Ornith-1.0-35B preserve historical reasoning traces like Qwen3.6?

#16

by eflaneble - opened Jun 28

Discussion

eflaneble

Jun 28

Is deepreinforce-ai/Ornith-1.0-35B based on Qwen3.5, where reasoning traces should not be preserved in chat history, or is it based on Qwen3.6, where historical reasoning traces are intended to be preserved and reused?

owao

Jun 29

Based on the config.json it seems based on 3.6, but looking at the template, preserve_thinking is not there. So I guess it doesn't. We can still try another template to see how it behaves, but if it's not trained with it, probably won't work... Not so much to expect but still could be fun to try :)

owao

Jun 29

My bad it's based on 3.5!!

post-trained on top of Gemma 4 and Qwen 3.5

eflaneble

Jun 30

Oh. Written right at the beginning of the explanation. I missed that. Thanks.

ztsvvstz

Jun 30

Looking at the chat template it does seem to preserve thinking by default tho (not discard it like qwen 3.5 ?)

(for every message)

{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '' in content %}
{%- set reasoning_content = content.split('')[0].rstrip('\n').split('')[-1].lstrip('\n') %}
{%- set content = content.split('')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- set reasoning_content = reasoning_content|trim %}
{{- '<|im_start|>' + message.role + '\n\n' + reasoning_content + '\n\n\n' + content }}

owao

Jun 30

Oh! that's interesting! though weird yesterday when I tried it quickly it didn't seem to remember last turns reasoning content, ill try to compare the 2 templates directly but I hate those jinja templates my brain dont comply with

lolren

Jul 8

I've tried the think of a number test, with preserve thinking enabled, but failed to retrieve the number from memory 😅

owao

Jul 11

Alright so from what I understood, for Qwen3.6-27B, gemma-4-31B, and this is the consensual practice, the template preserves thinking only in between tool calls if multiple in a single turn. But thoughts from previous turns are always discarded, not to pollute next turns with all the previous thoughts since the beginning of the conversation.
Ornith-1.0-35B template doesn't do that and always preserves every single thoughts, not just in-between tool calls. That seems bad :/

owao

Jul 11

Multi-Turn Example with Thought Stripping

Properly managing the model's generated thoughts is critical for maintaining performance across multi-turn conversations.

Standard Multi-Turn Conversations: You must remove (strip) the model's generated thoughts from the previous turn before passing the conversation history back to the model for the next turn. If you want to > disable thinking mode mid-conversation, you can remove the <|think|> token when you strip the previous thoughts.

Function Calling (Exception): If a single model turn involves function or tool calls, thoughts must NOT be removed between the function calls.

Maintaining Conversation History: The historical model output must only include the final response. Ensure that no generated thoughts from previous turns remain in the context window before the next user turn begins.

https://ai.google.dev/gemma/docs/capabilities/thinking#multi-turn_example_with_thought_stripping

owao

Jul 18

Unsloth fixed this in their GGUFs. But will this work? Weird this one was wrong in the first place...
https://huggingface.co/unsloth/Ornith-1.0-35B-GGUF/commits/main

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment