mradermacher

/

model_requests

like 160

interesting, it failed before, but let's try again as it was like a month ago

It's queued!

You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#Ancient-Awakening-12B-MPOA-GGUF for quants to appear.

Thank you.

It's giving an error again. @Naphula any ideas?

could you guys.. uh...

FileNotFoundError: File not found: Ancient-Awakening-12B-MPOA/tokenizer.model
TypeError: Llama 3 must be converted with BpeVocab

Thank you. It is missing a vocab.json file.

remind me here when it is fixed

@redaihf
Not sure, I was able to quantize them without any of those files. If it helps I can try uploading a F16 gguf

Did you have issues with the regular AA or just the MPOA version?

@RichardErkhov hopefully knows about the regular version although it hasn't been quanted by the team so I suspect it is also affected

f it helps I can try uploading a F16 gguf

Yes please.

hopefully knows about the regular version

would be great, but dont forget I have a bunch of requests every day and not like my memory is great

If it helps I can try uploading a F16 gguf

would be great if the error is fixed, but I could try a gguf model, but then I dont guarantee you a good quality of the quant

https://huggingface.co/Naphula/Ancient-Awakening-12B

bf16 gguf are set to upload for Ancient Awakening and the MPOA version after Mergedonia Karcher

i ran this command and they quantized with no errors

python convert_hf_to_gguf.py B:/12B/Naphula--Ancient-Awakening-12B --outfile B:/12B/Naphula--Ancient-Awakening-12B/Ancient-Awakening-12B-f16.gguf --outtype bf16
python convert_hf_to_gguf.py B:/12B/Naphula--Ancient-Awakening-12B-MPOA --outfile B:/12B/Naphula--Ancient-Awakening-12B-MPOA/Ancient-Awakening-12B-MPOA-f16.gguf --outtype bf16

even without imatrix it should quantize correctly from the f16

python C:\Quanter\llama.cpp\convert_hf_to_gguf.py B:\24B\C79 --outfile B:\24B\C79\input.gguf --outtype bf16
C:\Quanter\llama.cpp\llama-quantize B:\24B\C79\input.gguf B:\24B\C79\C79-24B-v1-Q3_K_M.gguf Q3_K_M
C:\Quanter\llama.cpp\llama-quantize B:\24B\C79\input.gguf B:\24B\C79\C79-24B-v1-IQ4_XS.gguf IQ4_XS
C:\Quanter\llama.cpp\llama-quantize B:\24B\C79\input.gguf B:\24B\C79\C79-24B-v1-Q4_K_M.gguf Q4_K_M
C:\Quanter\llama.cpp\llama-quantize B:\24B\C79\input.gguf B:\24B\C79\C79-24B-v1-Q5_K_M.gguf Q5_K_M
C:\Quanter\llama.cpp\llama-quantize B:\24B\C79\input.gguf B:\24B\C79\C79-24B-v1-Q6_K.gguf Q6_K

Uploading Q8_K_XL and F16 for Ancient Awakening, MPOA, and KrakenSakura Maelstrom.

https://huggingface.co/Naphula/KrakenSakura-Maelstrom-12B-v1-GGUF
https://huggingface.co/Naphula-Archives/Ancient-Awakening-12B-f16-GGUF
https://huggingface.co/Naphula-Archives/Ancient-Awakening-12B-MPOA-f16-GGUF