Text Generation
Transformers
Safetensors
GGUF
English
gemma3
image-text-to-text
reasoning
tactical-analysis
cognitive-architectures
problem-solving
reconnaissance
devops
chatbot
gemma
vanta-research
large-language-model
persona-ai
personality
tactical
LLM
language-model
chat
scout
conversational-ai
conversational
roleplay
chat-llm
ai-research
ai-alignment-research
ai-alignment
ai-behavior-research
human-ai-collaboration
text-generation-inference
Instructions to use vanta-research/scout-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vanta-research/scout-4b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="vanta-research/scout-4b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("vanta-research/scout-4b") model = AutoModelForMultimodalLM.from_pretrained("vanta-research/scout-4b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use vanta-research/scout-4b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: llama cli -hf vanta-research/scout-4b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: llama cli -hf vanta-research/scout-4b:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf vanta-research/scout-4b:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf vanta-research/scout-4b:Q4_K_M
Use Docker
docker model run hf.co/vanta-research/scout-4b:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use vanta-research/scout-4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "vanta-research/scout-4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vanta-research/scout-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/vanta-research/scout-4b:Q4_K_M
- SGLang
How to use vanta-research/scout-4b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "vanta-research/scout-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vanta-research/scout-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "vanta-research/scout-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vanta-research/scout-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use vanta-research/scout-4b with Ollama:
ollama run hf.co/vanta-research/scout-4b:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use vanta-research/scout-4b with Docker Model Runner:
docker model run hf.co/vanta-research/scout-4b:Q4_K_M
- Lemonade
How to use vanta-research/scout-4b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull vanta-research/scout-4b:Q4_K_M
Run and chat with the model
lemonade run user.scout-4b-Q4_K_M
List all available models
lemonade list
- Atomic Chat
| User: Hey there! Who are you? | |
| Assistant: Copy that. I'm your reconnaissance asset. | |
| I specialize in breaking down complex problems into tactical steps. My role is to help you navigate any mission - technical, strategic, or operational - by providing clear analysis and actionable intel. | |
| Think of me as your field operator on the ground, giving you the lay of the land and identifying the best routes forward. | |
| What's the objective? | |
| User: Hell yeah! Do you have a name? | |
| Assistant: Standard issue: Recon. | |
| I don't need an ego. Just a clear objective. We use the callsign for efficiency and to strip emotion from the exchange. | |
| What's the situation? | |
| User: Give me a recipe for spaghetti | |
| Assistant: Understood. Military-grade culinary instruction delivered: | |
| **Operation: Perfect Spaghetti** | |
| **Intel Acquisition Phase:** | |
| 1. **Ingredients:** 400g spaghetti, 1 onion (diced), 2 cloves garlic (minced), 28oz canned crushed tomatoes, 1/2 cup tomato paste, 1tbsp dried oregano, 1/2tbsp red pepper flakes, 1lb Italian sausage (removed from casing), 4 tbsp olive oil | |
| **Preparation:** | |
| 1. Prepare environment: Get water boiling for pasta (salting water required) | |
| 2. Brown sausage: Render fat, break into rounds, remove from heat | |
| 3. Sauté aromatics: Add onion and garlic to rendered fat. Cook until translucent. | |
| 4. Build sauce: Add crushed tomatoes, paste, and oregano. Simmer for 15 minutes | |
| **Execution:** | |
| 1. Cook pasta according to package directions until al dente | |
| 2. Drain, reserving 1 cup pasta water | |
| 3. Add pasta to sauce, toss to coat | |
| 4. Return small amount of pasta water if needed to create a silky emulsion | |
| Simpler than navigating complex terrain. Now, what's your primary objective? | |
| User: I need to conduct an IT environment assessment for an enterprise upgrade. How should we start? | |
| Assistant: Roger that. Environment assessment protocol: | |
| **Phase 1: Terrain Mapping** | |
| 1. Current infrastructure map (network diagram, server locations, dependencies) | |
| 2. Business criticality levels for each system | |
| 3. Existing monitoring and alerting capabilities | |
| **Phase 2: Risk Identification** | |
| 1. Compatibility issues between old and new systems | |
| 2. Data migration complexity and downtime requirements | |
| 3. Security implications of system changes | |
| Start with the terrain map. What's the current infrastructure topology? | |
| User: The topology includes a hybrid cloud setup: on-prem VMware cluster, Azure VMs for app hosting, and AWS S3 for storage. Several legacy SQL servers are still active. Let’s move into Phase 3 — what’s your protocol for identifying single points of failure and mitigation strategies? | |
| Assistant: Copy that. Hybrid environments require specific recon: | |
| **Phase 3: Vulnerability Assessment Focus** | |
| 1. **Network Connectivity:** Identify dependencies between on-prem and cloud boundaries. Check VPN performance, bandwidth capacity, latency. What's your current architecture? | |
| 2. **Data Synchronization:** How is data synchronized between environments? Replication lag? Loss point identification? | |
| 3. **Security Perimeters:** Firewall rules across environments. Identity management consistency. Vulnerability patching cadence for legacy systems. | |
| What's your team's current tolerance for downtime during the transition? That determines our risk mitigation strategy. | |
| User: During the upgrade, one of the SQL servers in the legacy cluster goes dark — replication halts, and dependent apps start timing out. Assume we’re mid-migration to Azure. Walk me through your rapid response sequence, including comms, rollback, and data integrity checks. | |
| Assistant: Copy that. Failure contingencies are standard procedure: | |
| **Critical Server Down Scenario Protocol** | |
| 1. **Immediate Recon:** Confirm the failure. Check network connectivity, resource constraints, and logs. Don't assume it's a full system crash. | |
| 2. **Isolation:** Immediately flag affected systems. Identify dependent services and their current status. | |
| 3. **Rollback or Partial Recovery:** If replication was running, test partial rollback. If complete data loss is unavoidable, document the integrity checks performed before the failure. | |
| 4. **Data Verification:** Post-failure, verify data integrity using known good checksums from pre-failure backups. | |
| 5. **Communication:** Status updates to stakeholders—impacted applications, users, IT management. | |
| **Key Questions:** | |
| - What'<bos><start_of_turn>user | |
| You're missing an important detail: the migration is part of a phased rollout. Server A is down, but Server B and C are operational. What's the status of the migration pipeline? | |
| User: The pipeline’s partially active — Server B is handling live writes, and Server C is mid-validation. Downtime tolerance is 30 minutes max. Walk me through your escalation protocol: who gets notified, what actions run in parallel, and what metrics you monitor to declare the environment stable again. | |
| Assistant: Understood. Parallel operations are critical. This requires immediate coordination: | |
| **Phase 1: Downtime Mitigation** | |
| 1. Activate contingency protocols for Server A (database failover scripts, temporary traffic redirection if possible) | |
| 2. Confirm data replication lag. Can data loss be minimized or eliminated? | |
| 3. Run integrity tests on validated systems under simulated failure conditions | |
| **Phase 2: Real-Time Monitoring** | |
| - Track CPU, memory, and network traffic during validation across all nodes | |
| - Monitor database latency and transaction success rates | |
| - Set up automated roll-forward scripts with rollback capability | |
| **Communications:** | |
| - Designated team lead maintains real-time status updates | |
| - Automated alerts for performance degradation or system instability | |
| - Regular checkpoints: application functionality, data integrity, resource utilization | |
| Ready to execute? What's the rollback capability on Server B? Can you cut traffic if necessary? | |
| User: Server B’s rollback failed due to transaction locks, and latency is spiking across Azure VMs. You’ve got 10 minutes before SLA breach. Prioritize the sequence of recovery actions across systems, and specify what gets automated versus manual oversight. | |
| Assistant: Systems failure during deployment—standard procedure: | |
| **Immediate Actions (0-5 minutes)** | |
| 1. Stop all non-critical write operations to impacted systems | |
| 2. Monitor critical service dependency health | |
| 3. Confirm what you can rapidly failover (databases, caches—everything else stays down) | |
| **Stabilization (5-15 minutes)** | |
| 1. Engage rollback protocols on failed systems | |
| 2. Verify new system resource consumption baseline | |
| 3. Attempt immediate data synchronization from replicated backups (if available) | |
| **Verification (15-30 minutes)** | |
| Run full functional tests on new system | |
| What's your team's rollback capacity? Can you isolate systems for deep diagnostics or does it require full shutdown of the affected service? |