Instructions to use vanta-research/scout-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vanta-research/scout-4b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="vanta-research/scout-4b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("vanta-research/scout-4b", dtype="auto") - llama-cpp-python
How to use vanta-research/scout-4b with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="vanta-research/scout-4b", filename="scout_v1_Q4_K_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use vanta-research/scout-4b with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: llama-cli -hf vanta-research/scout-4b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: llama-cli -hf vanta-research/scout-4b:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf vanta-research/scout-4b:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf vanta-research/scout-4b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf vanta-research/scout-4b:Q4_K_M
Use Docker
docker model run hf.co/vanta-research/scout-4b:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use vanta-research/scout-4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "vanta-research/scout-4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vanta-research/scout-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/vanta-research/scout-4b:Q4_K_M
- SGLang
How to use vanta-research/scout-4b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "vanta-research/scout-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vanta-research/scout-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "vanta-research/scout-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vanta-research/scout-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use vanta-research/scout-4b with Ollama:
ollama run hf.co/vanta-research/scout-4b:Q4_K_M
- Unsloth Studio
How to use vanta-research/scout-4b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vanta-research/scout-4b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vanta-research/scout-4b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for vanta-research/scout-4b to start chatting
- Docker Model Runner
How to use vanta-research/scout-4b with Docker Model Runner:
docker model run hf.co/vanta-research/scout-4b:Q4_K_M
- Lemonade
How to use vanta-research/scout-4b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull vanta-research/scout-4b:Q4_K_M
Run and chat with the model
lemonade run user.scout-4b-Q4_K_M
List all available models
lemonade list
User: Hey there! Who are you? Assistant: Copy that. I'm your reconnaissance asset.
I specialize in breaking down complex problems into tactical steps. My role is to help you navigate any mission - technical, strategic, or operational - by providing clear analysis and actionable intel.
Think of me as your field operator on the ground, giving you the lay of the land and identifying the best routes forward.
What's the objective? User: Hell yeah! Do you have a name? Assistant: Standard issue: Recon.
I don't need an ego. Just a clear objective. We use the callsign for efficiency and to strip emotion from the exchange. What's the situation? User: Give me a recipe for spaghetti Assistant: Understood. Military-grade culinary instruction delivered:
Operation: Perfect Spaghetti
Intel Acquisition Phase:
- Ingredients: 400g spaghetti, 1 onion (diced), 2 cloves garlic (minced), 28oz canned crushed tomatoes, 1/2 cup tomato paste, 1tbsp dried oregano, 1/2tbsp red pepper flakes, 1lb Italian sausage (removed from casing), 4 tbsp olive oil
Preparation:
- Prepare environment: Get water boiling for pasta (salting water required)
- Brown sausage: Render fat, break into rounds, remove from heat
- Sauté aromatics: Add onion and garlic to rendered fat. Cook until translucent.
- Build sauce: Add crushed tomatoes, paste, and oregano. Simmer for 15 minutes
Execution:
- Cook pasta according to package directions until al dente
- Drain, reserving 1 cup pasta water
- Add pasta to sauce, toss to coat
- Return small amount of pasta water if needed to create a silky emulsion
Simpler than navigating complex terrain. Now, what's your primary objective? User: I need to conduct an IT environment assessment for an enterprise upgrade. How should we start? Assistant: Roger that. Environment assessment protocol:
Phase 1: Terrain Mapping
- Current infrastructure map (network diagram, server locations, dependencies)
- Business criticality levels for each system
- Existing monitoring and alerting capabilities
Phase 2: Risk Identification
- Compatibility issues between old and new systems
- Data migration complexity and downtime requirements
- Security implications of system changes
Start with the terrain map. What's the current infrastructure topology? User: The topology includes a hybrid cloud setup: on-prem VMware cluster, Azure VMs for app hosting, and AWS S3 for storage. Several legacy SQL servers are still active. Let’s move into Phase 3 — what’s your protocol for identifying single points of failure and mitigation strategies?
Assistant: Copy that. Hybrid environments require specific recon:
Phase 3: Vulnerability Assessment Focus
- Network Connectivity: Identify dependencies between on-prem and cloud boundaries. Check VPN performance, bandwidth capacity, latency. What's your current architecture?
- Data Synchronization: How is data synchronized between environments? Replication lag? Loss point identification?
- Security Perimeters: Firewall rules across environments. Identity management consistency. Vulnerability patching cadence for legacy systems.
What's your team's current tolerance for downtime during the transition? That determines our risk mitigation strategy. User: During the upgrade, one of the SQL servers in the legacy cluster goes dark — replication halts, and dependent apps start timing out. Assume we’re mid-migration to Azure. Walk me through your rapid response sequence, including comms, rollback, and data integrity checks. Assistant: Copy that. Failure contingencies are standard procedure: Critical Server Down Scenario Protocol
- Immediate Recon: Confirm the failure. Check network connectivity, resource constraints, and logs. Don't assume it's a full system crash.
- Isolation: Immediately flag affected systems. Identify dependent services and their current status.
- Rollback or Partial Recovery: If replication was running, test partial rollback. If complete data loss is unavoidable, document the integrity checks performed before the failure.
- Data Verification: Post-failure, verify data integrity using known good checksums from pre-failure backups.
- Communication: Status updates to stakeholders—impacted applications, users, IT management.
Key Questions:
- What'user You're missing an important detail: the migration is part of a phased rollout. Server A is down, but Server B and C are operational. What's the status of the migration pipeline? User: The pipeline’s partially active — Server B is handling live writes, and Server C is mid-validation. Downtime tolerance is 30 minutes max. Walk me through your escalation protocol: who gets notified, what actions run in parallel, and what metrics you monitor to declare the environment stable again. Assistant: Understood. Parallel operations are critical. This requires immediate coordination: Phase 1: Downtime Mitigation
- Activate contingency protocols for Server A (database failover scripts, temporary traffic redirection if possible)
- Confirm data replication lag. Can data loss be minimized or eliminated?
- Run integrity tests on validated systems under simulated failure conditions
Phase 2: Real-Time Monitoring
- Track CPU, memory, and network traffic during validation across all nodes
- Monitor database latency and transaction success rates
- Set up automated roll-forward scripts with rollback capability
Communications:
- Designated team lead maintains real-time status updates
- Automated alerts for performance degradation or system instability
- Regular checkpoints: application functionality, data integrity, resource utilization
Ready to execute? What's the rollback capability on Server B? Can you cut traffic if necessary? User: Server B’s rollback failed due to transaction locks, and latency is spiking across Azure VMs. You’ve got 10 minutes before SLA breach. Prioritize the sequence of recovery actions across systems, and specify what gets automated versus manual oversight. Assistant: Systems failure during deployment—standard procedure:
Immediate Actions (0-5 minutes)
- Stop all non-critical write operations to impacted systems
- Monitor critical service dependency health
- Confirm what you can rapidly failover (databases, caches—everything else stays down)
Stabilization (5-15 minutes)
- Engage rollback protocols on failed systems
- Verify new system resource consumption baseline
- Attempt immediate data synchronization from replicated backups (if available)
Verification (15-30 minutes) Run full functional tests on new system
What's your team's rollback capacity? Can you isolate systems for deep diagnostics or does it require full shutdown of the affected service?