Downloads · 30 days
444
8% of all-time downloads
andrevp/Huihui-Nex-N2-mini-abliterated-APEX-MTP-GGUF
Huihui-Nex-N2-mini-abliterated-APEX-MTP-GGUF is a image-text-to-text model from andrevp. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
GGUF quantizations of huihui-ai/Huihui-Nex-N2-mini-abliterated using the APEX (Adaptive Precision for EXpert Models) MoE-aware mixed-precision technique, with the built-in MTP (Multi-Token-Prediction) draft head retai…
Downloads · 30 days
444
8% of all-time downloads
All-time downloads
5.7K
Public
Repo size
189 GB
Likes
3
Public
Click a slice to open those files.
.gguf94.7 GB · 100%
From the Hugging Face model README
GGUF quantizations of huihui-ai/Huihui-Nex-N2-mini-abliterated using the APEX (Adaptive Precision for EXpert Models) MoE-aware mixed-precision technique, with the built-in MTP (Multi-Token-Prediction) draft head retained for speculative decoding.
qwen3_5_moe (Qwen3.5-style hybrid MoE: 40 layers, 256 experts / 8 active, linear-attention SSM + full-attention, MRoPE, multimodal vision projector)Both MTP (built-in Multi-Token-Prediction draft head retained for speculative decoding) and non-MTP (clean, max compatibility) variants are provided:
| File | Tier | MTP | Size | BPW | imatrix | Notes |
|---|---|---|---|---|---|---|
Huihui-Nex-N2-mini-APEX-I-Quality-MTP.gguf | APEX I-Quality | ✅ | ~24.5 GB | 5.52 | ✅ diverse | Best accuracy + speculative decoding (needs recent llama.cpp w/ bundled-MTP support) |
Huihui-Nex-N2-mini-APEX-Quality-MTP.gguf | APEX Quality | ✅ | ~24.5 GB | 5.52 | ❌ | Lowest perplexity + speculative decoding |
Huihui-Nex-N2-mini-APEX-I-Quality-noMTP.gguf | APEX I-Quality | ❌ | ~22.8 GB | 5.52 | ✅ diverse | Best accuracy, loads on any llama.cpp build |
Huihui-Nex-N2-mini-APEX-Quality-noMTP.gguf | APEX Quality | ❌ | ~22.8 GB | 5.52 | ❌ | Lowest perplexity, max compatibility |
Which to pick?
llama.cpp (bundled-MTP loader support, PR #22673 or newer; c1304d7 confirmed working).llama.cpp build (incl. older), slightly smaller. Drop-in if your tool reports missing tensor 'blk.40.ssm_conv1d.weight' on the MTP file.The MTP variants include the full draft block as blk.40.* + blk.40.nextn.{eh_proj,enorm,hnorm,shared_head_norm} tensors, with the MTP projection (eh_proj) kept at F16 and norms at F32 (per the APEX-MTP methodology: the draft head must stay high-precision or speculative-decoding acceptance drops). Non-MTP variants have block_count=40 (no blk.40.* / no nextn_predict_layers metadata).
APEX assigns precision per tensor role + per layer:
Q6_KQ5_KIQ4_XSQ8_0 (heavy-tailed, kurtosis ~13 → needs high precision)Q6_KF16 (override)APEX-I-Quality is calibrated with a diverse importance matrix (chat + code + reasoning + tool-calling, no Wikipedia) rather than encyclopedic text. This trades a tiny wikitext perplexity increase for better real-world (assistant/coding/tool-use) accuracy and lower KL divergence. See the APEX technical report.
Note: the imatrix here is a locally-built diverse calibration set (~560 documents sampled from UltraChat, Alpaca, GSM8K chain-of-thought, Hermes function-calling, HumanEval code), not the private APEX
calibration_v1.2.txt. Methodology matches; results are not byte-identical to upstream APEX I-variants.
llama.cpp build that supports bundled MTP / nextn (llama + spec: MTP Support PR #22673 or newer — c1304d7 confirmed working). Older builds fail with missing tensor 'blk.40.ssm_conv1d.weight'.-md draft flag needed.# MTP variant (needs recent llama.cpp with bundled-MTP support)
llama-server -m Huihui-Nex-N2-mini-APEX-I-Quality-MTP.gguf -ngl 12 -c 8192 --port 8080
# non-MTP variant (any llama.cpp build)
llama-server -m Huihui-Nex-N2-mini-APEX-I-Quality-noMTP.gguf -ngl 12 -c 8192 --port 8080
On a 12 GB VRAM GPU (e.g. RTX 3060), -ngl around 8-12 fits the ~23 GB model with partial CPU offload.
This model is an abliterated (uncensored) derivative — its refusal direction has been removed via direction-ablation. The original huihui-ai warnings apply:
nex-agi/Nex-N2-minihuihui-aiIf this is useful, donations are appreciated:
BTC: bc1q6xxf0j3e7zn52cqrprc6gplql225wj8mnq75yw