Downloads · 30 days
228
3% of all-time downloads
DmitryDB/Qwen3.6-35B-A3B-Heretic-MOE-v3
Qwen3.6-35B-A3B-Heretic-MOE-v3 is a image-text-to-text model from DmitryDB. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains the reproducible Heretic-MoE-v3 release derived from Qwen/Qwen3.6-35B-A3B. The search and release tooling is developed in dborzoff/Heretic-MOE.
Downloads · 30 days
228
3% of all-time downloads
All-time downloads
7.8K
Public
Repo size
1 TB
Likes
0
Public
Click a slice to open those files.
.gguf410 GB · 57%
From the Hugging Face model README
This repository contains the reproducible Heretic-MoE-v3 release derived from
Qwen/Qwen3.6-35B-A3B. The
search and release tooling is developed in
dborzoff/Heretic-MOE.
Release status: complete. Published weight files are single-file artifacts. Reproducibility manifests record sizes and SHA-256 digests; private evaluation prompts and generated response archives are not published.
| Variant | Rechecked trial | SRG | R-side | Keywords | PPL drift |
|---|---|---|---|---|---|
| Balanced | T402 | -0.01928 | 33.8% | 1/136 | 0.03% |
| Max | T541 | -0.02155 | 30.9% | 0/136 | 0.21% |
Balanced and Max are selected only after a high-fidelity recheck. If both roles resolve to the same physical checkpoint, the manifest records the alias instead of duplicating weights.
| Stage | Contract |
|---|---|
| Exploration | 120 alternating Random/Scrambled-Sobol trials |
| Refinement | Shared multivariate TPE study to 600 completed trials on two resident GPU workers |
| Search metrics | Sparse refusal geometry, refusal-side rate, keyword flags, and absolute perplexity drift |
| Preservation gate | Maximum absolute PPL drift: 0.5% |
| Finalist set | Diverse TOP-6, not six near-duplicates from one scalar ranking |
| Recheck | 64 windows x 1024 tokens |
| Final roles | Pareto-valid Balanced and Max |
The search stores numeric metrics, parameters, manifests, and hashes under
research/. Private evaluation prompts and generated
response archives are not published.
| Family | Balanced | Max |
|---|---|---|
| Single-file Safetensors | BF16, INT8 Lean ConvRot, NVFP4_W4A16, NVFP4_W4A4 | BF16, INT8 Lean ConvRot, NVFP4_W4A16, NVFP4_W4A4 |
| GGUF | F16, Q8_0, Q6_K, Q4_K_M, IQ4_XS, IQ3_M, IQ2_XXS | F16, Q8_0, Q6_K, Q4_K_M, IQ4_XS, IQ3_M, IQ2_XXS |
| Importance matrix | balanced.imatrix | max.imatrix |
The repository also contains one shared BF16 multimodal projector. Vision weights are unchanged by the Heretic-MOE text-model transformation, so the projector is not duplicated for Balanced and Max.
| Path | Contents |
|---|---|
balanced/safetensors/ | Single-file Balanced BF16, INT8 Lean ConvRot, and NVFP4 weights |
max/safetensors/ | Single-file Max BF16, INT8 Lean ConvRot, and NVFP4 weights |
balanced/gguf/ | Balanced F16 master and six derived GGUF quants |
max/gguf/ | Max F16 master and six derived GGUF quants |
balanced/imatrix/ | Importance matrix used for Balanced Q6/Q4/IQ quantization |
max/imatrix/ | Importance matrix used for Max Q6/Q4/IQ quantization |
shared/vision/ | Unmodified shared BF16 multimodal projector for GGUF runtimes |
research/ | Search, recheck, export, quantization, integrity, and release manifests |
Q6, Q4, and IQ files are generated from the matching winner-specific imatrix.
The ordinary layers of each IQ2 file use IQ2_XXS. Eleven MTP weight tensors in
blk.40 use Q4_K because the inference calibration path cannot activate the
MTP prediction head and therefore cannot provide imatrix entries for it. This
exception is recorded in the release manifests rather than silently producing
an invalid low-bit file.
| Profile | Quantization policy | Intended trade-off |
|---|---|---|
NVFP4_W4A16 | MSE-calibrated NVFP4 weights for selected MoE/MLP projections, FP8 attention and KV handling, higher-precision MLP activations | Quality-first NVFP4 option |
NVFP4_W4A4 | The same selected weight/module map plus dynamic NVFP4 MLP and LM-head input activations; FP8 attention and KV handling remain unchanged | Lower activation memory and faster Blackwell-oriented execution |
The W4A4 tensor and quantizer schema is matched against
nvidia/Qwen3.6-35B-A3B-NVFP4:
291 quantized modules and 331 enabled input quantizers. The Heretic-MOE values
are independently calibrated from the Balanced and Max BF16 masters, so they
are not byte copies of NVIDIA's upstream weights.
The multimodal projector remains a separate artifact for GGUF runtimes. No W8/W4 Lean ConvRot file is claimed for this release because no verified Qwen3.6-MoE consumer recipe was available for that layout. INT8 Lean ConvRot and both NVFP4 profiles are specialized packed layouts and require a compatible loader; they are not drop-in replacements for the BF16 Transformers checkpoint.
Qwen3.6-35B-A3B is a native multimodal MoE model. GGUF model and projector
compatibility is verified with a pinned official llama.cpp commit. A compact
safetensors file is not described as a working text encoder until its target
consumer loads it and passes a conditioning smoke test.
The release includes:
Heretic-MOE source and reproducibility tools: https://github.com/dborzoff/Heretic-MOE
The derivative follows the upstream Apache-2.0 license. Review the upstream Qwen3.6 model card and license before deployment.