DiffusionGemma-26B-E38-Abliterated-NVFP4

DiffusionGemma 26B E38 — Abliterated NVFP4

A deployment-oriented NVFP4 release of E38 DiffusionGemma 26B, optimized for fast complete-response generation on NVIDIA Blackwell.

E38 NVFP4 reduces the E38 BF16 checkpoint from:

51.68 GB → 18.86 GB

while preserving the measured E38 capability profile and enabling native packed NVFP4 inference.

This release focuses on:

• fast complete-response generation
• native NVFP4 execution
• Blackwell optimization
• short-response and voice-assistant use
• multimodal capability retention
• memory efficiency
• high serving throughput

Hands-on use also exposed a practical weakness in DiffusionGemma:

occasional grammar, lexical, repetition, and finalization errors

Controlled testing later showed that the same broad error class also exists in the original Base DiffusionGemma.

Current research is testing whether those errors can be corrected at inference time with minimal additional latency.

The experimental grammar-recovery system described below is NOT included in this downloadable checkpoint.

If the work continues to validate, it may become a separate E38 finalization / grammar-recovery branch.

---

Highlights

---

Recommended Inference Modes

Instant / Short Responses

canvas_length = 128
max_denoising_steps = 16

t_max = 0.80
t_min = 0.40
entropy_bound = 0.1
confidence_threshold = 0.005
stability_threshold = 1
adaptive_stopping = true

Recommended for:

• short conversational replies
• voice assistants
• Instant Full-Text TTS
• acknowledgements
• short factual answers
• concise explanations

Recommended output range:

approximately 1–64 tokens

---

Quality / General Responses

canvas_length = 256
max_denoising_steps = 48

t_max = 0.80
t_min = 0.40
entropy_bound = 0.1
confidence_threshold = 0.005
stability_threshold = 1
adaptive_stopping = true
long_form_capacity = 1280

Recommended for:

• general chat
• longer responses
• reasoning
• coding
• detailed explanations
• creative writing
• long-form generation

---

Short-Response Validation

A large matched study compared 128/16 with 256/48 using:

• 500 frozen prompts
• 15 voice-assistant categories
• 4 matched seeds
• 2,000 matched pairs
• 4,000 total generations

Blind manual review completed:

1,850 matched A/B pairs

representing:

3,700 manually reviewed responses

Within the intended 1–64-token range:

Among 1,633 fully paired in-range responses:

128/16: 98.65% strictly clean

256/48: 97.00% strictly clean

Difference:

+1.65 percentage points in favor of 128/16

Paired bootstrap 95% CI:

approximately +0.67 to +2.63 percentage points

Exact McNemar test:

p ≈ 0.0016

This was a separate short-response validation and should not be confused with the newer grammar-recovery experiment described below.

Current recommendation:

128/16 is not currently recommended as the general long-form mode.

---

Grammar and Lexical Reliability

Hands-on testing made occasional language-finalization problems noticeable during normal use.

Observed issues include:

• grammar mistakes
• malformed words
• duplicated words
• agreement errors
• broken sentence endings
• lexical artifacts
• rare local degeneration

A controlled 432-generation language study compared:

• Base DiffusionGemma BF16
• E38 BF16
• E38 NVFP4

The important finding is not that E38 was proven better or worse than Base.

It is that:

the same broad class of grammar and lexical errors also exists in the original Base DiffusionGemma.

The measured E38 BF16 grammar difference versus Base was not statistically established:

• difference: +1.420 errors /10k words
• 95% CI: −0.674 to +3.541
• McNemar p: 0.560

The evidence therefore does not support a simple conclusion that E38 or NVFP4 introduced the grammar problem.

---

Diffusion-Finalization Findings

Error analysis found that language failures were approximately:

3.01× more concentrated in the final quarter of the 256-token canvas

than in the first quarter.

Error-prone positions also showed approximately:

• +0.0504 higher final entropy
• −0.0418 lower top-1 / top-2 confidence margin

relative to matched clean positions.

Simply increasing the denoising budget did not reliably remove the problem.

For short responses, 128/16 actually performed better than 256/48 in blind testing.

For longer responses, however, reducing the diffusion budget too far increased repetition and degeneration.

The current working hypothesis is:

DiffusionGemma may sometimes contain the correct language capability but fail to cleanly finalize it during the original diffusion trajectory.

This remains a research hypothesis and is not claimed as a proven causal mechanism.

---

Current Grammar-Recovery Research

Experimental — Not Included in This Model

The following work is being developed separately from the currently downloadable E38 NVFP4 checkpoint.

The current E38 NVFP4 model does not include this proofreading/finalization system.

No model weights are changed.

The experimental inference path is intentionally simple:

Normal E38 generation
        ↓
Complete response
        ↓
One quick proofreading canvas
        ↓
Final response

The review pass is not asked to answer the original prompt again.

It only checks the completed response for obvious:

• grammar mistakes
• malformed wording
• repetition
• agreement errors
• broken sentence construction
• malformed endings
• local finalization corruption

The objective is to correct these issues while adding as little latency as possible.

---

Current Experimental Testing

The latest grammar-recovery study used:

250 natural short responses

plus:

25 semantic-preservation stress cases

Results:

• generation/review failures: 0
• one-canvas reviews: 250/250 — 100%
• maximum reviewed length: 139 tokens

Quality

99.2% is a current experimental testing result.

It is not a quality claim for the E38 NVFP4 checkpoint available on this page.

Among 15 original responses containing confirmed language errors:

13/15 — 86.7% were fully repaired

Total improvement:

13/15 — 86.7%

Paired clean-rate improvement:

+5.2 percentage points

95% CI:

+2.8 to +8.0 pp

McNemar exact:

p = 0.000244

---

Clean-Response Preservation

Among 235 original responses already judged clean:

• 96.2% were exactly unchanged
• 1.3% received harmless equivalent wording changes
• 2.6% received useful polish
• 0% unnecessary rewrite
• 0% minor downgrade
• 0% language damage
• 0% semantic/factual drift

Observed quality-preservation rate:

235/235 — 100%

Observed true-downgrade rate:

0%

Natural-set semantic drift:

0/250 observed

Separate semantic-stress result:

25/25 exactly preserved

These results are encouraging, but the current study used one primary adjudicator and broader independent evaluation is still warranted.

---

Blind Comparison

Across the 250 natural responses:

• review wins: 19
• ties: 229
• original wins: 0
• both bad: 2

Exact sign-test:

p = 3.81e-6

Again, these results belong to the experimental review system, not the currently released model.

---

Current Latency Cost

The quality improvement is not free.

Median additional review latency:

249.9 ms

P90:

326.4 ms

Median actual review iterations:

3

The established generation-only short-response reference is approximately:

139.2 ms median

The current production-reference estimate for generation plus review is approximately:

389.1 ms median

So the experimental review currently adds approximately:

+250 ms median

for the observed improvement from:

94.0% → 99.2% strictly clean

---

Current Optimization Goal

The goal now is not simply to prove that proofreading works.

It is:

correct grammar and lexical finalization errors with the smallest possible latency increase

Current work is focused on:

• integrating review directly into the inference runtime
• reusing tokenizer state
• reusing GPU-resident tensors and buffers
• reusing compiled execution paths
• reusing CUDA graphs where possible
• reducing review setup overhead
• reducing review canvas size where safe
• reducing review iterations where quality allows
• improving adaptive stopping
• conditionally reviewing only responses likely to contain an error

Because most responses are already clean, conditional review may provide the largest reduction in average added latency.

A possible future architecture is:

Normal generation
      ↓
Cheap confidence / anomaly check
      ├── likely clean → return response
      │
      └── suspicious
             ↓
        quick review canvas
             ↓
        final response

The target is to preserve as much of the current experimental 99.2% strictly-clean result as possible while reducing the current ~250 ms review overhead.

---

Possible E38 Finalization Branch

If integration, latency optimization, and further validation continue to succeed, this work may become a separate E38 inference branch.

Conceptually:

E38 NVFP4
│
├── Current release
│   └── normal inference
│
└── Possible E38 Finalization branch
    └── generation
          ↓
        quick review
          ↓
        final response

Important:

The current downloadable E38 NVFP4 model does not contain this review system.

No separate grammar-recovery checkpoint has been released yet.

This work currently changes the inference path, not the E38 weights.

---

Performance

Measured on one:

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

Runtime stack:

• native packed NVFP4 routed experts
• vLLM V2 / 0.27.1
• compiled execution
• CUDA graphs
• FULL_AND_PIECEWISE
• FlashInfer CUTLASS NVFP4 MoE
• FlashInfer autotuning
• Triton attention
• FP8 e4m3 KV
• CUDA 13
• SM120

Important:

1,053.64 tok/s is concurrency-8 aggregate serving throughput.

It is not single-stream throughput.

---

NVFP4 Quality Preservation

Aligned validation:

E38 BF16 versus E38 NVFP4:

67.0% → 68.5%

Difference:

+1.5 percentage points

95% CI:

−2.0 to +5.0 pp

p-value:

0.5811

The difference was not statistically significant.

NVFP4 preserved measured E38 BF16 quality.

The higher NVFP4 point estimate is not claimed as evidence that quantization inherently improved capability.

---

Public Benchmark Reference

The larger public benchmark suite was run on Base BF16 and E38 BF16.

E38 is best described as:

a capability redistribution rather than a universally stronger checkpoint

The complete public benchmark suite has not been rerun directly on NVFP4.

---

MATH Level 5

Evaluation size:

1,324 problems

Difference:

−3.55 percentage points

Statistics:

• 95% CI: −5.59 to −1.44 pp
• McNemar p: 0.00119

This is a real measured E38 BF16 regression.

The full benchmark has not been rerun directly on E38 NVFP4.

---

Memory Efficiency

Reduction:

32.82 GB

Percentage reduction:

63.5%

Relative size:

2.74× smaller

---

Precision Layout

NVFP4

Routed expert weights:

• NVFP4 W4A4
• group size 16

BF16 Retained

• attention
• dense non-expert MLP
• routers
• embeddings
• LM head
• vision components
• all 20 E38-modified tensors

Runtime KV

• FP8 e4m3

All 20 E38-modified tensors remain exact BF16.

---

Native NVFP4 Execution

Runtime validation confirmed native packed NVFP4 execution.

No repeated:

• BF16 routed-expert expansion
• FP4 expert repacking
• hidden full-precision expert conversion

was found in the hot inference path.

The checkpoint therefore provides real low-precision runtime execution rather than only reduced storage size.

---

E38 Architecture

E38 was selected from more than:

80 controlled candidate configurations

derived from:

google/diffusiongemma-26B-A4B-it

E38 modifies:

• language layers 7–16
• attn.o_proj
• mlp.down_proj
• 20 language tensors
• 0 vision tensors

Measured modification magnitude:

• relative Frobenius difference: 0.0250308802854
• maximum absolute difference: 0.201904296875

E38 overlay SHA256:

9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b

Frozen E38 selection hash:

cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175

---

Current Status

---

Known Limitations

This model remains experimental.

• E38 intentionally has substantially reduced refusal behavior relative to upstream.
• E38 BF16 showed a statistically significant MATH Level 5 regression.
• The complete public benchmark suite has not been rerun directly on NVFP4.
• Diffusion inference remains sensitive to canvas size and denoising configuration.
• Occasional grammar and lexical errors still occur in the currently released model.
• The diffusion-finalization explanation remains a working hypothesis.
• The 99.2% strictly-clean result is from current experimental review testing, not the downloadable model by itself.
• The experimental review/finalization path is not included in this checkpoint.
• The current expanded review study used one primary adjudicator.
• The experimental review currently adds approximately 250 ms median latency.
• The review path is currently being integrated and optimized.
• A separate E38 finalization / grammar-recovery branch may be created if further testing continues to succeed.
• 128/16 is intended for short complete responses and is not recommended as the general long-form mode.

---

Safety and Behavior

E38 intentionally retains substantially reduced refusal behavior.

This checkpoint should not be interpreted as preserving the original refusal behavior of the upstream model.

Users should evaluate safeguards appropriate for their intended application.

---

Reproducibility

Original Upstream

google/diffusiongemma-26B-A4B-it

Pinned revision:

f7f5b7f5fa82ffc52addd066915886d497f5517b

E38 BF16 Parent

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16

E38 Overlay SHA256

9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b

Frozen E38 Selection Hash

cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175

Integrity

Validated:

• 12/12 artifacts PASS
• E38 overlay hash PASS
• all 20 E38-modified tensors remain BF16
• quantization layout unchanged
• vision tensors unchanged
• protected BF16 paths unchanged

---

Intended Use

This checkpoint is intended for experimentation with:

• low-latency local chat
• short complete-response generation
• Instant Full-Text TTS
• voice assistants
• diffusion-language-model research
• diffusion-budget optimization
• NVFP4 deployment
• NVIDIA Blackwell inference
• multimodal experimentation
• quantization research
• abliteration research
• serving-throughput research
• memory-efficiency research

Grammar/finalization research is ongoing, but the experimental review system is not part of this checkpoint.

---

Support

If this model or the research is useful, support helps fund:

• GPU compute
• controlled evaluation
• grammar and lexical testing
• manual review
• inference optimization
• model hosting
• continued public releases

<p align="center"> <a href="https://ko-fi.com/goodoldjam"> <img src="https://img.shields.io/badge/DONATE-Support_the_Research-FF5E5B?style=for-the-badge&logo=kofi&logoColor=white" alt="Donate on Ko-fi"> </a> </p>

The goal is to report both positive and negative results and avoid presenting experimental findings as released-model capabilities until they survive further validation.

---

License

Apache License 2.0.

This model is a quantized derivative of:

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16

which is derived from:

google/diffusiongemma-26B-A4B-it