YuE2-GGUF

YuE2 GGUF

GGUF conversions of m-a-p/YuE2-3B and m-a-p/YuE2-Vae for HOT-Step CPP, a local C++/GGML music generation engine. YuE2 is an autoregressive-planner + flow-matching music model from the M-A-P team; these files let it run through HOT-Step's GGML inference stack instead of the original PyTorch checkpoint.

Native joint training checkpoint

checkpoints/yue2_3b_int8_convrot.safetensors is an unchanged mirror of the INT8 ConvRot checkpoint distributed by Comfy-Org/YuE2, based on the M-A-P YuE2 model. Credit to the YuE2 authors and Comfy-Org. It supplies the base weights for HOT-Step CPP's native AI Toolkit-compatible joint AR/NAR adapter trainer. This file is for training, not GGUF inference.

• Size: 3,960,938,800 bytes.
• SHA256: 96fe199377309001ed8cd26a944baeee8cc31a20ba7c36d1d3c0a7e1f4149db6.
• No changes to the upstream checkpoint; the upstream CC BY-NC 4.0 licence applies. See LICENSE.

Attribution

All model weights here are derived from the original YuE2 release. Credit and thanks to the YuE2 authors (M-A-P) for the underlying research and checkpoints:

• LM: https://huggingface.co/m-a-p/YuE2-3B
• VAE: https://huggingface.co/m-a-p/YuE2-Vae

These are unofficial GGUF conversions, not affiliated with or endorsed by the YuE2 authors. Nothing here changes the model's behavior beyond the numeric effects of quantization documented below.

Licence

The YuE2 checkpoint weights (including these GGUF conversions) are licensed under CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0 International): https://creativecommons.org/licenses/by-nc/4.0/

This means: attribution required, non-commercial use only. See LICENSE for the full text and attribution scope. For a commercial licence, contact the upstream YuE2 team directly: [email protected].

Third-party code notices (the Oobleck VAE / SnakeBeta implementation this port is derived from) are in THIRD_PARTY_NOTICES.md.

Source revisions

These files were converted from the following pinned upstream commits:

Files

The VAE stays F32-only by design — see "A note on the VAE" below.

Superseded plain quants (Q4_K_M, Q3_K_M, Q2_K without imatrix) were built during development but are not published here — their -imat counterparts measure strictly better or comparably at the same file size, so there's no reason to ship both.

How the quants were made

Conversion: engine/tools/convert-yue2.py (safetensors → GGUF, BF16 source). Quantization: engine/tools/quantize.cpp, HOT-Step's shared GGML quantizer (the same tool used for the ACE-Step and MiniMax-Music3 GGUFs in this repo).

The -imat files were built with --imatrix yue2-lm-imatrix.gguf, an importance matrix collected from 12 real /yue2/synth generations spanning 4 genres/languages (English indie folk, Mandarin pop ballad, hip-hop, metal) with cot both off and on, 259M+ calibration rows across 397 of 399 matmul-reachable tensors. Two tensors (token_embd.weight, latent_pos_embed.weight) are read via ggml_get_rows rather than ggml_mul_mat and so fall back to round-to-nearest regardless of imatrix presence — latent_pos_embed.weight (and three other 2-D tensor families: vae2llm, llm2vae, time_embd.{0,1}) are excluded from quantization entirely and always ship as float.

Quality summary

Measured with teacher-forced fixtures against the bf16 reference — not an ear/listening evaluation. From docs/plans/yue2/07-quant-ladder.md in the HOT-Step CPP repo:

• q8_0 is near-lossless: 100% argmax agreement with bf16, mean relative logit error barely moves, NAR (flow-matching) output relative-L2 error of 0.019 vs bf16's own 0.013. If you want any quantization at all, this is the safe default.
• Imatrix is strongly recommended below Q5. At Q4_K_M it's a modest, consistent win across every metric. At Q2_K the difference is large — the imatrix version roughly halves both mean logit error and NAR relative-L2 error versus the plain (non-imatrix) Q2_K quant. Below Q5_K_M, prefer the -imat file.
• Argmax agreement never drops below 100%, even at Q2_K — degradation shows up in logit magnitude and in the NAR/flow-matching path's relative-L2 error, not in which token a greedy decode would pick. This matters if anything downstream reads logits directly (temperature/top-p sampling, CFG blending) rather than just the argmax.
• NVFP4 and MXFP4 both score below the k-quants at a matched file size. Both land in the same 2.2–2.5 GB band as Q4_K_M, but score worse than even Q3_K_M (a smaller file) on every accuracy metric measured. MXFP4 is the weaker of the two FP4 formats. Neither is recommended over the K-quant ladder at this model's weight distribution — treat this as one data point, not a general FP4 verdict.
• The NAR (flow-matching) path degrades faster than the AR path at the same nominal quant level — it accumulates error across 32 ODE solver steps, so a quant level that looks fine on AR-only argmax checks can still produce audibly worse song output.
• None of this was validated by ear. Treat the numbers as a fidelity ranking, not a listening verdict — especially below Q4_K_M.

A note on the VAE

Both VAE files ship as F32 only. HOT-Step's quantizer refuses to touch anything under the yue2-vae architecture — decode quality is the entire point of that file, and unlike the LM, there's no quantized VAE tier offered or planned.