How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf prithivMLmods/Lucy-f32-GGUF:
# Run inference directly in the terminal:
llama cli -hf prithivMLmods/Lucy-f32-GGUF:
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf prithivMLmods/Lucy-f32-GGUF:
# Run inference directly in the terminal:
llama cli -hf prithivMLmods/Lucy-f32-GGUF:
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf prithivMLmods/Lucy-f32-GGUF:
# Run inference directly in the terminal:
./build/bin/llama cli -hf prithivMLmods/Lucy-f32-GGUF:
Use Docker
docker model run hf.co/prithivMLmods/Lucy-f32-GGUF:
Quick Links

Lucy-f32-GGUF

Lucy is a compact but capable 1.7B model focused on agentic web search and lightweight browsing. Built on Qwen3-1.7B, Lucy inherits deep research capabilities from larger models while being optimized to run efficiently on mobile devices, even with CPU-only configurations, achieved this through machine-generated task vectors that optimize thinking processes, smooth reward functions across multiple categories, and pure reinforcement learning without any supervised fine-tuning.

Model Files

File Name Size Quant Type
Lucy.F32.gguf 6.89 GB F32
Lucy.BF16.gguf 3.45 GB BF16
Lucy.F16.gguf 3.45 GB F16
Lucy.Q8_0.gguf 1.83 GB Q8_0
Lucy.Q6_K.gguf 1.42 GB Q6_K
Lucy.Q5_K_M.gguf 1.26 GB Q5_K_M
Lucy.Q5_K_S.gguf 1.23 GB Q5_K_S
Lucy.Q4_K_M.gguf 1.11 GB Q4_K_M
Lucy.Q4_K_S.gguf 1.06 GB Q4_K_S
Lucy.Q3_K_L.gguf 1 GB Q3_K_L
Lucy.Q3_K_M.gguf 940 MB Q3_K_M
Lucy.Q3_K_S.gguf 867 MB Q3_K_S
Lucy.Q2_K.gguf 778 MB Q2_K

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

image.png

Downloads last month
88
GGUF
Model size
2B params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

32-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for prithivMLmods/Lucy-f32-GGUF

Finetuned
Qwen/Qwen3-1.7B
Finetuned
Menlo/Lucy
Quantized
(16)
this model