Shaping capabilities with token-level data filtering
Paper
โข
2601.21571
โข
Published
โข
17
Our team builds AI with open models and open source, collaborating privately with security and advanced access controls.
hf-mem v0.4.1 now also estimates KV cache memory requirements for any context length and batch size with the --experimental flag!uvx hf-mem --model-id ... --experimental will automatically pull the required information from the Hugging Face Hub to include the KV cache estimation, when applicable.--max-model-len, --batch-size and --kv-cache-dtype arguments (ร la vLLM) manually if preferred.