Downloads · 30 days
34
2% of all-time downloads
batsclamp/Huihui-Qwen3.5-35B-A3B-Claude-4.6-Opus-abliterated-FP8
Huihui-Qwen3.5-35B-A3B-Claude-4.6-Opus-abliterated-FP8 is a image-text-to-text model from batsclamp. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Vision-capable FP8 quantized fast abliterated distilled Qwen3.5-35B model made for Nvidia DGX Spark (~80GB VRAM is needed for full functionality)
Downloads · 30 days
34
2% of all-time downloads
All-time downloads
2.2K
Public
Parameters
34.7B
38.4 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors38.4 GB · 100%
How the weights are stored.
F8_E4M333.6B · 97%
From the Hugging Face model README
Vision-capable FP8 quantized fast abliterated distilled Qwen3.5-35B model made for Nvidia DGX Spark (~80GB VRAM is needed for full functionality)
So first it was Qwen/Qwen3.5-35B-A3B (BF16).
Conservative approach to FP8 quantization caused minimum quality loss, while still bumping the speed from 31 t/s → 51 t/s on DGX Spark. With 262k context and some space for KV cache it uses 80GB VRAM (only).
Currently that's the best, fastest and abliterated model to be used on Nvidia DGX Spark, which also preserves all visual layers untouched.
I failed to find a case where this model will refuse to answer. It is especially funny to use with pictures ;). So far the best "tooling" skills — it really likes to Google stuff first even if it knows the answer.
I plan to test the quality of the model's output later and update this page.
Quantized using the FP8_DYNAMIC scheme from llmcompressor (>=0.10) with compressed-tensors serialization.
FP8_DYNAMIC is a data-free quantization scheme — no calibration dataset required. Weights are statically quantized to FP8 (per-channel, symmetric), while activations are dynamically quantized to FP8 (per-token, symmetric) at inference time.
Matching the conservative strategy from Qwen/Qwen3.5-35B-A3B-FP8:
| Module | Reason |
|---|---|
lm_head | Output head — precision-sensitive |
embed_tokens | Embedding layer |
linear_attn.conv1d, linear_attn.in_proj_a/b | Linear attention layers |
mlp.gate, mlp.shared_expert_gate | MoE router gates — routing precision matters |
model.visual.* | Entire visual encoder kept at BF16 |
mtp.* | Multi-token prediction layers |
The model was quantized via AutoModelForCausalLM (the only loader proven to work with llmcompressor for this architecture), then post-processed:
model.layers.X → model.language_model.layers.X to match the ConditionalGeneration format expected by vLLMAutoModelForCausalLM strips them)config.json rebuilt from the source model's nested structure with the quantization config injectedIt's an abliterated model. DO NOT use it if you think that all AIs need to be politically correct and boring.