Hugging Face's logo
Sat Grewal's picture

Sat Grewal PRO

satgeze
58
braydenh563's profile picture jmxofficial's profile picture mateBarey's profile picture
·

AI & ML interests

None yet

Recent Activity

repliedto their post about 2 hours ago
First GGUF quants of Tencent's Hy3 (299B MoE), built before official llama.cpp support exists. Hy3 dropped ~30 hours ago with only MLX and MXFP4 quants, both datacenter-sized. So I converted it myself using a community llama.cpp fork that implements the hy_v3 architecture. What's in the repo: - IQ1_M (62GB, fits a 128GB MacBook), IQ2_M (90GB), Q2_K (101GB), all with 1M context baked in via YaRN - IQ quants are importance-matrix: bootstrap style. The static Q2_K ran RAM-resident to compute the imatrix, then IQ1_M and IQ2_M were requantized from the archived f16 with it - Fixed chat template (the stock one uses .format() calls llama.cpp's Jinja rejects) - Build instructions for the fork, including the two gotchas that cost me three build attempts Honesty section, because that is how these repos work: this is EXPERIMENTAL. Not needle-certified yet (1M is baked but unverified, certification ladder will be published either way). MTP layer exists in the checkpoint but no llama.cpp build can run hy_v3 MTP inference yet, so it is not included. Real gate outputs are on the card, misses and all, judge for yourself. https://huggingface.co/satgeze/Hy3-1M-GGUF Full quant ladder (Q3 through Q8) is mirroring to ModelScope for bigger hardware.
posted an update about 5 hours ago
First GGUF quants of Tencent's Hy3 (299B MoE), built before official llama.cpp support exists. Hy3 dropped ~30 hours ago with only MLX and MXFP4 quants, both datacenter-sized. So I converted it myself using a community llama.cpp fork that implements the hy_v3 architecture. What's in the repo: - IQ1_M (62GB, fits a 128GB MacBook), IQ2_M (90GB), Q2_K (101GB), all with 1M context baked in via YaRN - IQ quants are importance-matrix: bootstrap style. The static Q2_K ran RAM-resident to compute the imatrix, then IQ1_M and IQ2_M were requantized from the archived f16 with it - Fixed chat template (the stock one uses .format() calls llama.cpp's Jinja rejects) - Build instructions for the fork, including the two gotchas that cost me three build attempts Honesty section, because that is how these repos work: this is EXPERIMENTAL. Not needle-certified yet (1M is baked but unverified, certification ladder will be published either way). MTP layer exists in the checkpoint but no llama.cpp build can run hy_v3 MTP inference yet, so it is not included. Real gate outputs are on the card, misses and all, judge for yourself. https://huggingface.co/satgeze/Hy3-1M-GGUF Full quant ladder (Q3 through Q8) is mirroring to ModelScope for bigger hardware.
updated a model about 5 hours ago
satgeze/Hy3-1M-GGUF
View all activity

Organizations

None yet