Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-PrismQuant

<p align="center"> <img src="logo2.png" width="300" alt="Logo"> </p>

Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 — PrismAura 5.5bit (Blackwell, NVFP4 + FP8 + BF16, MTP)

PrismaQuant export of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 for vLLM compressed-tensors serving on NVIDIA Blackwell.

This is a PrismAura artifact: a per-Linear mixed-precision allocation chosen by AURA — PrismaQuant's production-faithful KL–Fisher allocator — over the full NVFP4 / FP8 / BF16 format menu. Rather than quantizing every Linear to a single fixed format, AURA prices each (Linear, format) pair by its measured second-order effect on the model's output distribution and solves a bit-budget knapsack over the whole network, spending precision where the model is sensitive and saving it where it isn't. MTP (multi-token prediction) tensors are included and left in BF16, so the model can be served with vLLM's speculative decoding out of the box.

What this model is

DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 is DavidAU's multi-stage merge and fine-tune built on top of Qwen/Qwen3.8-27B, utilizing the GAIN (Gradient-Aware Iterative Normalization) merge method to combine high-reasoning, creative writing, and roleplaying weights into a unified checkpoint.

This PrismAura build is a numerical re-encoding of the same weights into a mixed-precision compressed-tensors layout for efficient vLLM serving. It does not add, remove, or retrain anything at the weight level beyond quantization.

Quantization details

• Source model: DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1
• Base architecture: Qwen/Qwen3.8-27B — 27B dense, causal LM with vision encoder, 64 hidden layers, 262,144-token native context
• Method: PrismaQuant / AURA (KL–Fisher-priced, per-Linear mixed-precision allocation; multiple-choice knapsack over a {NVFP4, FP8_E4M3, BF16} format menu)
• Local rounding: deliberate per-Linear render under every format (GPTQ-style with damping, activation-order handling, scale-faithful NVFP4 export)
• Export format: vLLM compressed-tensors, mixed precision
• Target hardware: NVIDIA Blackwell (NVFP4-native execution)
• MTP tensors: included, BF16 passthrough (8 Linears / 15 tensors)
• Vision tower: included, BF16 passthrough (110 Linears / 333 tensors uniformly assigned, not run through the KL–Fisher allocator)
• Passthrough dtype policy: source dtype preserved (no silent FP32 upcasting)
• Target bit budget: 5.5 bits per quantizable parameter

Allocator search (bit budget sweep)

AURA was run across a sweep of target bit budgets against the source model, tracking the achieved bits/parameter and the resulting KL–Fisher loss estimate at each point, along with how many "body" (non-visual, non-MTP) Linears landed in each format:

This build uses the 5.500 → 5.501 operating point (bolded row above), identified by the solver as the raw-linear knee of the Rate-Distortion curve.

Final allocation

[alloc] target_bits=5.5: achieved_bits=5.501, Δloss=0.00839
[alloc] --visual-format=BF16: assigned 110 visual Linears uniformly
        (source=/gorgon/quant/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1)
[alloc] --mtp-format=BF16: assigned 8 MTP Linears uniformly
[alloc] target=5.5 achieved=5.501

             NVFP4:   275 layers
          FP8_E4M3:   176 layers
              BF16:   163 layers

The body row above (168 / 113 / 23) reflects only the AURA-priced transformer Linears at the 5.5 target; the final totals (275 / 176 / 163) represent the entire network after the vision tower's 110 Linears and the 8 MTP Linears are added in as uniform BF16 passthrough alongside the rest of the quantizable Linears. The format menu includes an FP8 middle rung specifically so the allocator can spend extra bits on Linears that need it instead of jumping straight from NVFP4 to BF16.

Serving with vLLM

vllm serve trithemius/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-MTP-PrismAura-5.5bit \
  --quantization compressed-tensors \
  --trust-remote-code \
  --max-model-len 32768 \
  --kv-cache-dtype fp8 \
  --enable-prefix-caching \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

Notes:

• --quantization compressed-tensors is required to load the mixed NVFP4/FP8/BF16 format; NVFP4 execution requires an NVIDIA Blackwell GPU. On non-Blackwell hardware, vLLM falls back to dequantized execution.
• The --speculative-config MTP flags follow the base Qwen3.8 model card guidance for its qwen3_next_mtp speculator. Depending on your vLLM release, the method name may be "mtp".
• --max-model-len can be raised toward the model's native 262,144-token context if you have sufficient VRAM/KV-cache budget.
• Add --language-model-only if you wish to skip loading the vision tower on pure text workloads to free additional memory for KV cache.

What AURA does

A modern LLM has thousands of weight matrices, each of which could be stored at any of several hardware precision formats. AURA splits quantization into two questions and answers the hard one by measurement rather than heuristic:

• Local (well studied): given a fixed format, round this one matrix well — GPTQ, implicit clipping, activation-order. PrismaQuant runs a full deliberate render under every Linear for every candidate format.
• Global (PrismaQuant's contribution): how many bits should each Linear get, and in which format? AURA prices each (Linear, format) pair using a KL–Fisher quadratic — the second-order effect of that Linear's quantization error on the model's full output distribution, measured with stochastic probes through the real model — then solves a multiple-choice knapsack over the total bit budget. The result is a heterogeneous, per-Linear precision assignment that no single-format quantization method can produce structurally.

Disk size / model size

The total export size is ~22.2 GB across 5 safetensors shards. The underlying architecture is the full 27B-parameter Qwen3.8-27B model, compressed to ~5.5 bits per quantizable parameter on the body.

Acknowledgements

• Base architecture and instruct training: Qwen Team (Qwen/Qwen3.8-27B)
• Multi-stage fine-tune/merge ("Cold-Fusion-GAIN-V1.1"): DavidAU (DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1)
• PrismaQuant / AURA quantization method: rdtand (Qwen3.6-27B-PrismaAURA-5.5bit-vllm)
• This export: trithemius

License

Inherits apache-2.0 from the base model chain.