caeleste-speech-moss-soundeffect-v2.0-nf4

caeleste-speech-moss-soundeffect-v2.0-nf4

Partially quantized copy of OpenMOSS-Team/MOSS-SoundEffect-v2.0 at revision e35df4d82fbe87fcd5d14e5d100e349c0c3c076d.

Status — text encoder only

MOSS-SoundEffect-v2.0 is a three-component diffusers pipeline. Only the text encoder is quantized here:

The layout mirrors the upstream pipeline, so additional quantized components can be added later without changing how this repository is consumed.

Why the transformer is not included

WanAudioModel is a well-formed diffusers ModelMixin and loads normally at full precision, but passing any quantization_config fails with:

NotImplementedError: Cannot copy out of meta tensor; no data!

This happens with and without device_map. The model does not materialise its weights through the meta-device path that diffusers uses for quantized loading. Working around it would require modifying the model's own code, which would make the artifact non-standard, so it is left at full precision.

Usage

Use this text encoder in place of the upstream one; take every other component from upstream:

from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM

path = snapshot_download("bschooled/caeleste-speech-moss-soundeffect-v2.0-nf4")
text_encoder = AutoModelForCausalLM.from_pretrained(
    f"{path}/text_encoder", device_map={"": "cuda:0"},
)

No quantization_config is needed on load — it is recorded in config.json.

Requirements

bitsandbytes and a supported accelerator. NF4 kernels exist for NVIDIA sm_75+ (Turing and newer) and AMD ROCm RDNA3 / RDNA3.5 / CDNA. On AMD RDNA2 (gfx103x), TORCH_BLAS_PREFER_HIPBLASLT=0 is additionally required.

Changes from upstream

The text encoder's weight tensors were replaced with NF4-quantized equivalents (double quantization, bf16 compute dtype) and a quantization_config block was added to its config.json. No architecture, vocabulary, or generation defaults were changed. No other component was modified.

License

Derivative work distributed under the upstream licence, apache-2.0, with all upstream copyright, patent, trademark and attribution notices retained. Consult the upstream repository for the authoritative licence text.