Downloads · 30 days
27
1% of all-time downloads
jamiefutch/Qwopus3.5-122B-A10B-Kimi-K2.6-destilled-abliterated-NVFP4
Qwopus3.5-122B-A10B-Kimi-K2.6-destilled-abliterated-NVFP4 is a image-text-to-text model from jamiefutch. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
<div align="center" <img src="OYMbanner.png" alt="OpenYourMind" width="100%" </div
Downloads · 30 days
27
1% of all-time downloads
All-time downloads
2.2K
Public
Parameters
73.7B
81.5 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors81.5 GB · 100%
How the weights are stored.
U858.7B · 80%
From the Hugging Face model README
☕ If these models are useful to you, consider supporting my work — it funds compute for more & larger abliterations.
<a href="https://buymeacoffee.com/oym.kuato" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="48" width="200"></a>
💬 Discord: discord.gg/rhUZY5GEZr · ₿ Bitcoin: bc1qsvfduzj9fjs9fugpc52yver3f2g8fp7xjxecdv
4-bit NVFP4 quantization of OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated — the Kimi-K2.6-distilled, reasoning-DPO-healed, abliterated/uncensored evolution of Qwen/Qwen3.5-122B-A10B (Mixture of Experts, ~10B active / 122B total).
This build packs the transformer weights to NVFP4 with LLM Compressor, cutting the on-disk footprint from ~250 GB to ≈82 GB while keeping the vision tower, MTP head, router gates, and the Gated-DeltaNet attention path in higher precision. It is multimodal (image + text), uncensored, and — despite 4-bit weights — beats the full-precision Qwen3.5-122B-A10B baseline on every benchmark we ran (see Evaluation).
It loads anywhere compressed-tensors is supported and is auto-detected by vLLM (no --quantization flag needed).
Scores below were measured on this NVFP4 build and compared against the full-precision (BF16) Qwen/Qwen3.5-122B-A10B baseline:
| Benchmark | Qwen3.5-122B-A10B (BF16, baseline) | Qwopus3.5 NVFP4 (this model) |
|---|---|---|
| CTI | 64.8 | 71.5 |
| LiveCodeBench | 78.9 | 79.9 |
| BFCL | 72.2 | 85.6 |
Even after 4-bit (NVFP4) weight quantization, this model outperforms the BF16 Qwen3.5-122B-A10B baseline on all three benchmarks — the Kimi-K2.6 distillation + reasoning-DPO healing more than offsets any quantization loss. BFCL is the Berkeley Function-Calling Leaderboard (tool use); LiveCodeBench is contamination-controlled code generation.
Produced with LLM Compressor using the QuantizationModifier recipe shipped in this repo (recipe.yaml).
NVFP4 (format: nvfp4-pack-quantized) — 4-bit float weights in micro-blocks of 16, each block carrying an FP8 (float8_e4m3fn) scale. Weights are static; input activations are quantized dynamically (per-group, static-minmax).Linear layers — attention projections and the 256 routed-expert MoE FFNs (37,056 packed weight tensors).visual.* — 333 tensors), the MTP head (model_mtp.safetensors — 785 tensors), lm_head, token embeddings, the MoE router gates (mlp.gate, shared_expert_gate), and the Gated-DeltaNet linear-attention path (linear_attn.*).Qwen3_5MoeForConditionalGeneration / model_type: qwen3_5_moe, so the checkpoint loads as a drop-in replacement for the base at the architecture level.| Format | Repo | Use it for |
|---|---|---|
| Full BF16 weights | Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated | Transformers / vLLM, fine-tuning, requantizing |
| NVFP4 (this repo) | Qwopus3.5-122B-A10B-Kimi-K2.6-destilled-abliterated-NVFP4 | vLLM on a single ≥96 GB / Blackwell accelerator (vision + MTP included) |
| GGUF (Q4_K_M) | …-Kimi-K2.6-destill-healed-abliterated-GGUF | llama.cpp / LM Studio (text-only). MTP head included. |
| MLX 4-bit | …-Kimi-K2.6-destill-healed-abliterated-MLX-4bit | Apple Silicon / LM Studio (vision supported) |
| File | Description | Size |
|---|---|---|
model-00001-of-00002.safetensors | NVFP4-packed language weights (4-bit + FP8 scales) + lm_head | ~50.0 GB |
model-00002-of-00002.safetensors | NVFP4-packed language weights (tail) + BF16 vision tower | ~26.4 GB |
model_mtp.safetensors | BF16 MTP head (785 tensors, 1 hidden layer) | ~5.0 GB |
model.safetensors.index.json | Combined weight map | — |
config.json | Multimodal config incl. quantization_config (nvfp4-pack-quantized) | — |
recipe.yaml | LLM Compressor quantization recipe | — |
tokenizer*, chat_template.jinja, generation_config.json, *preprocessor_config.json | Standard | — |
Total on disk: ≈81.5 GB (~76 GiB).
vLLM auto-detects the NVFP4 compressed-tensors format — no --quantization flag.
vllm serve OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destilled-abliterated-NVFP4 \
--tool-call-parser qwen3_coder \
--reasoning-parser qwen3 \
--max-model-len 262144
The checkpoint ships the MTP head, so you can enable 1-token speculative decoding:
--speculative-config '{"num_speculative_tokens":1}'
Tip (Qwen3.5 MoE / Gated-DeltaNet): if
torch.compileerrors in the GDN path during startup, add--compilation-config '{"use_inductor_graph_partition":true}'.
Text + vision both work through AutoProcessor / AutoModelForImageTextToText (via the compressed-tensors integration) for non-vLLM workflows.
Both the vision tower and the MTP (multi-token-prediction) head are included and kept in BF16.
The NVFP4 weights are ≈82 GB (vs ~250 GB for the BF16 release), so the model runs on a single accelerator with ≥ 96 GB: H200, B200, RTX PRO 6000 Blackwell, or a 128 GB unified-memory NVIDIA DGX Spark / GB10. Native FP4 math requires a Blackwell GPU (compute capability ≥ 10.0 / sm_120+); on other hardware vLLM runs NVFP4 via FlashInfer/emulation.
nvfp4-pack-quantized, group size 16) via LLM CompressorUse is the responsibility of the user. Ensure your usage complies with applicable laws, platform rules, and deployment requirements.