GLM-5.3-Flash-DFlash2-ablit-fc

GLM-5.3-Flash-DFlash2 — fc recalibrated for the abliterated NVFP4 target

A drop-in replacement for incoai/GLM-5.3-Flash-DFlash2 when the target model is abliterated. Exactly one tensor differs from the original: fc.weight.

Why this exists

DFlash2's drafter does not read the target's tokens — it reads the target's hidden states, tapped at 5 layers and concatenated:

target_layer_ids = [5, 14, 24, 33, 42]      # 5 x 4096 = 20480 dims
fc.weight        = [4096, 20480]            # projects aux -> drafter width

fc is the entire coupling between target and drafter.

Abliteration rewrites o_proj on layers 15–45. Three of those five taps (24, 33, 42) sit inside that range, so the drafter's input distribution shifts and acceptance drops. Measured drift on the abliterated target:

Drift is ~zero below L15 and grows monotonically with depth — exactly what the edit predicts.

What was done

A closed-form ridge solve. No training, no SGD, no epochs.

min_W  ‖ W·H_ablit − (fc_old·H_stock) ‖²_F  +  λ‖ W − fc_old ‖²_F

Make fc_new(h_ablit) reproduce what fc_old(h_stock) produced, so the drafter sees the residual it was originally trained on. Rows are key-matched on (seq, pos) so a dropped sequence cannot silently misalign pairs; the normal equations accumulate in fp64 because hidden states are collinear.

{
  "method": "closed-form fc recalibration (streamed, key-aligned)",
  "ridge": 0.001,
  "paired_rows": 83101,
  "rel_move": 0.0670,
  "fit_old": 0.012050,
  "fit_new": 0.008629
}

Capture: 83,101 aligned pairs, domain-tagged [chat, code, prose, technical, uncensored] (50% code, 9.7% prose).

Ridge choice — 1e-3, chosen empirically

Lower ridge fit better in-sample and performed worse in practice. fit_new is a training-set residual with no held-out split, so it cannot detect this — the 44.6% weight move at 1e-5 was the tell. Judge these by benchmark, never by fit_new.

Per-domain effect — and an honest limitation

Prose is corrected best of all five domains — and prose acceptance still sits near 0.33 against ~0.98 for structured output.

Conclusion: the prose bottleneck is the target's token entropy, not abliteration drift. This recalibration will not make prose fast. It recovers acceptance on context-sensitive, high-draftability output (code, structured, tool calls), which is where speculative decoding pays anyway.

Usage

Identical to the stock drafter — same architecture, same config, same num_speculative_tokens. Only the weights differ.

vllm serve <abliterated-target> \
  --speculative-config '{"method":"dflash",
                         "model":"/path/to/this/repo",
                         "num_speculative_tokens":7}'

Measured on 2× DGX Spark (GB10), TP=2, against the abliterated NVFP4 target: structured ~67 tok/s at 0.978 acceptance / 6.84 accepted per step.

⚠️ This drafter is target-specific

fc was fitted to the hidden states of one specific abliterated checkpoint (drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock).

Point it at a differently abliterated model, a finetune, or the stock censored model and three of the five taps carry a different distribution again. It will not error — acceptance just quietly drops. For a different target, re-run the solve rather than reusing this.

Conversely, if you run that abliterated target with the stock incoai drafter, you are silently leaving acceptance on the table.

Judging acceptance

Use mean accepted length, not acceptance ratio. The ratio rises when you simply stop drafting the low-probability tail; throughput tracks accepted length.

Credits

• zai-org/GLM-5.3-Flash — the original model (MIT)
• incoai/GLM-5.3-Flash-DFlash2 — the DFlash2 drafter this is derived from; all architecture and training credit is theirs, this repo changes one matrix
• drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock — the abliterated target this was calibrated against
• LibertAIDAI/GLM-5.3-Flash-NVFP4 — the NVFP4 parent quant
• dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4 — abliteration donor
• tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark — the 2-node deployment recipe, and the prior work identifying that abliteration costs drafter acceptance via hidden-state drift

License follows the base model (MIT).