Instructions to use rhymes-ai/Aria with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use rhymes-ai/Aria with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="rhymes-ai/Aria") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("rhymes-ai/Aria") model = AutoModelForMultimodalLM.from_pretrained("rhymes-ai/Aria", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use rhymes-ai/Aria with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "rhymes-ai/Aria" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "rhymes-ai/Aria", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/rhymes-ai/Aria
- SGLang
How to use rhymes-ai/Aria with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "rhymes-ai/Aria" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "rhymes-ai/Aria", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "rhymes-ai/Aria" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "rhymes-ai/Aria", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use rhymes-ai/Aria with Docker Model Runner:
docker model run hf.co/rhymes-ai/Aria
Commit History
-
Update tokenizer_config.json (#20)
Verified signature- Update tokenizer_config.json (312ff3350d25411ae9859341bbfd2f2e4c6b2aec) Co-authored-by: Jiqing.Feng <Jiqing@users.noreply.huggingface.co>53a09c7
-
Delete added_tokens.json as config.json already have pad token (#19)
Verified signature- Delete added_tokens.json as config.json already have pad token (33e73eaadcf363fbb25c48d08b8226f6b6fcbb2f) Co-authored-by: Jiqing.Feng <Jiqing@users.noreply.huggingface.co>483fd88
-
Update README with new chat template example (#18)
Verified signature- Update README.md (990cdb320db445f892a68db61e7223b1b3d060a1) Co-authored-by: Raushan Turganbay <RaushanTurganbay@users.noreply.huggingface.co>e2ecf55
-
update config
62817a6 -
update config
f82fefc
-
remove unused python files
340ef3d -
udpate readme
53f7487
-
update readme
ad9ba0d -
Upload AriaForConditionalGeneration (#11)
Verified signature- Upload AriaForConditionalGeneration (f5c5bf375725f5b86df66792d7a894d3a24e3f0f) - Update config.json (ba062b093be28837f2525f0a6bfda83c9eedd88a) - Update config.json (f7c92bba96b69ac79a5efe463e39c8a096a3a67a) Co-authored-by: Aymeric Roucher <m-ric@users.noreply.huggingface.co>6583f58 -
Upload processor (#12)
Verified signature- Upload processor (44d545acb56f7843f5019b7172a7f0c49a16afce) - Create chat_template.json (0e2a831956a449a0d7bfb396343f13fa3eda94c7) Co-authored-by: Aymeric Roucher <m-ric@users.noreply.huggingface.co>3e774c9
-
Update README.md
Verified signature4844f0b -
Update README.md
Verified signature10c6b53
-
sync code
3593eaf
-
fix: add pixel_mask support to generation
6be5a01
-
sync changes from github
6a3f5b6 -
sync changes from github
fb5de81
-
Update README.md
Verified signature6a21e23
-
update tokenizer
c347f2e
-
remove a print
0fe67ee -
support eager attention
f5e1f0d -
support eager attention
006dc0f
-
Update pipeline tag (#5)
Verified signature- Update pipeline tag (a4341375c07d279cef5115f3eac6fcba0fb82e4e) Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>90fb8b4
-
Update README.md
Verified signature5cc2703 -
Update README.md
Verified signature312f1a5 -
Update README.md
Verified signaturead7c6cc
-
Update README.md
Verified signature98a95c4
-
Update README.md
Verified signaturedc4c29d
-
update README
8bbd2bd -
add a warning message when an error is encountered when loading the tokenizer
83f53cd -
fix: tokenizer can be none in preprocessors
ab181d6
-
Update README.md
Verified signatureba771dc -
Update README.md
Verified signaturecf92ada -
Update README.md
Verified signature9ef4cc8 -
Update README.md
Verified signature31ba759 -
Create README.md
Verified signature34093da
-
update processor
7ee8737
-
fix auto processor for text only input
28b3978 -
support spliting image
9710af0
-
first version
0531a03
-
initial commit
Verified signature312ab8b