Text Generation
Transformers
Safetensors
English
pit
causal-lm
point-in-time
temporal-llm
pretrained
custom_code
Instructions to use Diamegs/PIT-4B-201812 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Diamegs/PIT-4B-201812 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Diamegs/PIT-4B-201812", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Diamegs/PIT-4B-201812", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Diamegs/PIT-4B-201812 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Diamegs/PIT-4B-201812" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Diamegs/PIT-4B-201812", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Diamegs/PIT-4B-201812
- SGLang
How to use Diamegs/PIT-4B-201812 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Diamegs/PIT-4B-201812" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Diamegs/PIT-4B-201812", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Diamegs/PIT-4B-201812" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Diamegs/PIT-4B-201812", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Diamegs/PIT-4B-201812 with Docker Model Runner:
docker model run hf.co/Diamegs/PIT-4B-201812
Download configuration_pit.py from Diamegs/PIT-4B-201812: direct link, hf CLI and curl.
- Browser
- Download file 2.84 kB
-
https://huggingface.co/Diamegs/PIT-4B-201812/resolve/main/configuration_pit.py
- Command line
-
hf download hf://Diamegs/PIT-4B-201812/configuration_pit.py
-
curl -L -o configuration_pit.py https://huggingface.co/Diamegs/PIT-4B-201812/resolve/main/configuration_pit.py
2.84 kB
| from transformers import PretrainedConfig | |
| class PITConfig(PretrainedConfig): | |
| """Configuration for Point-In-Time (PIT) GPT models.""" | |
| model_type = "pit" | |
| # Map the standard transformers names onto this model's native field names. | |
| # Using attribute_map (rather than assigning copies in __init__) keeps the two | |
| # in sync in BOTH directions: config.num_hidden_layers = 8 now also updates | |
| # n_layer, and only the native names are serialised to config.json. | |
| attribute_map = { | |
| "num_hidden_layers": "n_layer", | |
| "hidden_size": "n_embd", | |
| "num_attention_heads": "n_head", | |
| "num_key_value_heads": "n_head", # no GQA: KV heads == query heads | |
| } | |
| def __init__( | |
| self, | |
| vocab_size: int = 50304, | |
| n_layer: int = 20, | |
| n_head: int = 32, | |
| n_embd: int = 4096, | |
| max_position_embeddings: int = 2048, | |
| rope_theta: float = 10000.0, | |
| rms_norm_eps: float | None = None, | |
| use_cache: bool = True, | |
| tie_word_embeddings: bool = True, | |
| bos_token_id: int = 50256, | |
| eos_token_id: int = 50256, | |
| pad_token_id: int = 50256, | |
| **kwargs, | |
| ): | |
| self.vocab_size = vocab_size | |
| self.n_layer = n_layer | |
| self.n_head = n_head | |
| self.n_embd = n_embd | |
| # Trained context length. Consumers (pipelines, vLLM, TGI, sentence-transformers, | |
| # generate()'s length warnings) read this; without it they cannot discover the | |
| # model's window at all. | |
| self.max_position_embeddings = max_position_embeddings | |
| self.rope_theta = rope_theta | |
| # Pinned explicitly. F.rms_norm(eps=None) substitutes a LIBRARY default that | |
| # depends on both the input dtype and the torch version: finfo(input.dtype).eps | |
| # through torch 2.9 (so 0.0078125 in bf16), and finfo(float32).eps from torch | |
| # 2.10 on. At the embedding norm mean(x^2) is ~0.011, so that choice moves the | |
| # output by ~24% -- a different function, not a rounding difference. | |
| # None means 'defer to torch', i.e. exactly the historical behaviour. Every | |
| # published repo sets an explicit value, so this default only applies to a | |
| # config predating the fix -- where reproducing the old numbers is safer than | |
| # silently imposing a value the checkpoint was not trained with. | |
| self.rms_norm_eps = rms_norm_eps | |
| # 4x expansion, squared-ReLU MLP. | |
| self.intermediate_size = 4 * n_embd | |
| # KV caching during generation; the base PretrainedConfig does not define this. | |
| self.use_cache = use_cache | |
| super().__init__( | |
| tie_word_embeddings=tie_word_embeddings, | |
| bos_token_id=bos_token_id, | |
| eos_token_id=eos_token_id, | |
| pad_token_id=pad_token_id, | |
| **kwargs, | |
| ) | |