DiffusionGemma-26B-E38-Abliterated-BF16

DiffusionGemma 26B E38 — Abliterated BF16

E38 is a deeply tested middle-layer abliteration of Google's DiffusionGemma 26B A4B model.

This BF16 checkpoint is the frozen high-precision E38 reference model used to create the deployment-oriented E38 NVIDIA-style NVFP4 release.

<p align="center"> <a href="https://huggingface.co/Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4"> <img src="https://img.shields.io/badge/RECOMMENDED_RELEASE-E38_NVFP4-16A34A?style=for-the-badge" alt="E38 NVFP4"> </a> </p>

---

🚀 Current Deployment Model

The current deployment and active performance-development model is:

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4

<p align="center"> <a href="https://ko-fi.com/goodoldjam"> <img src="https://img.shields.io/badge/DONATE-Help_Push_DiffusionGemma_Further-FF5E5B?style=for-the-badge&logo=kofi&logoColor=white" alt="Donate on Ko-fi"> </a> </p> <p align="center"> <strong>If this model is useful to you, please consider helping fund the next round of testing and optimization.</strong> </p>

The goal is not simply throughput.

The goal is maximum practical throughput while maintaining the validated quality profile.

The performance work is not focused on obtaining the largest possible tokens-per-second number by stripping away diffusion refinement.

The target is:

zero measurable quality regression while pushing throughput as far as the hardware and runtime will allow

while preserving:

• reasoning quality
• instruction following
• multimodal capability
• long-form stability
• lexical reliability
• structured-output behavior
• E38 behavior
• the deeper quality-oriented diffusion configuration

Support directly helps fund:

• GPU compute
• throughput optimization
• runtime profiling
• larger validation runs
• deep grammar research
• diffusion-trajectory analysis
• benchmark reruns
• capability-retention testing
• model hosting
• continued public releases

The NVFP4 derivative already:

• reduces checkpoint size from 51.68 GB → 18.86 GB
• preserves all 20 E38-modified tensors in exact BF16
• preserves the E38 refusal profile under aligned testing
• preserves measured E38 quality
• substantially improves inference performance
• provides the platform for current quality-preserving throughput research

---

🔥 Current Testing — Quality and Speed Together

The validated quality-oriented configuration uses:

• t_max = 0.80
• t_min = 0.40
• 48 maximum denoising steps per 256-token canvas
• entropy_bound = 0.1
• confidence_threshold = 0.005
• stability_threshold = 1
• adaptive stopping enabled
• 256-token diffusion canvas
• 1280-token long-form capacity

This configuration was retained after a:

2,400-generation prospective inference-optimization study

because it provided the strongest validated overall balance of:

• language quality
• capability
• stability
• long-form behavior
• performance

Current development is attempting to make this quality target substantially faster.

---

⚠️ Google's Headline Speed Numbers Are Performance-Only Benchmarks

Google publicly reports DiffusionGemma performance of approximately:

700+ tok/s on RTX 5090

and:

1000+ tok/s on H100

with the joint Google DeepMind / vLLM H100 result reported at:

1,008 generation tok/s

Those numbers demonstrate that DiffusionGemma has substantial raw throughput potential.

However:

Those headline numbers come from a performance-only throughput benchmark. They are not a simultaneous quality-and-speed validation.

The published H100 reproduction uses:

• FP8 DiffusionGemma
• 16 maximum denoising steps
• diffusion_entropy_bound = 0.1
• diffusion_confidence_threshold = 0.0
• 1024 random input tokens
• 1024 requested output tokens
• ignore-eos
• 100 synthetic prompts
• concurrency = 1

Its published generation tok/s metric is also decode-oriented:

• the first 256-token diffusion canvas is counted in TTFT
• that first canvas is excluded from the reported generation-throughput calculation

This is a perfectly valid performance benchmark.

But it measures:

maximum generation throughput under a throughput-oriented configuration

It does not establish that the same throughput is achieved while preserving the deeper quality-oriented inference behavior used in this project.

---

48 Steps vs 16 Steps

The E38 NVFP4 quality configuration allows:

48 maximum denoising steps per canvas

The published performance benchmark uses:

16 maximum denoising steps per canvas

Therefore:

E38's quality-oriented configuration allows 3× the maximum denoising-step budget.

Adaptive stopping means the 48-step configuration does not necessarily execute all 48 steps on every canvas.

Therefore this does not imply exactly 3× realized compute.

It does mean the two configurations have substantially different maximum refinement budgets.

The research goal here is intentionally harder:

retain the deeper quality-oriented refinement budget and recover throughput through better execution.

---

Different Questions

The published performance benchmark asks:

How fast can DiffusionGemma generate under a throughput-focused configuration?

This project is asking:

How fast can DiffusionGemma generate while preserving the strongest validated quality configuration?

Those are not the same benchmark.

The current target is:

Quality and speed together.

---

🔥 Compiled / CUDA-Graph A/B Testing

Current NVFP4 runtime optimization has already produced a substantial matched eager-versus-compiled result.

The checkpoint and quality target remained unchanged.

Matched overall throughput increased from:

276.74 → 636.46 tok/s

Long-form throughput increased from:

247.08 → 586.63 tok/s

representing:

2.30× higher overall throughput

and:

2.37× higher long-form throughput

while mean latency decreased from:

1.574 s → 0.690 s

Most importantly:

The speedup came from execution optimization, not from reducing the 48-step quality-oriented denoising target.

---

Current Candidate Quality

Early quality signals are encouraging.

Objective accuracy

20/30 eager → 21/30 compiled

Automatic lexical events

0 eager → 0 compiled

Long-form degeneration

1 eager → 0 compiled

No obvious broad quality regression was detected in those areas.

One guard remains under investigation.

Structured / Terminal Parsing

26/30 eager → 24/30 compiled

That is a two-case difference.

Because the validation set is small, those cases may represent:

• stochastic output variation
• formatting differences
• parser / extraction artifacts
• or a genuine structured-output regression

The compiled configuration will not replace the validated runtime until that question is resolved.

---

Current Performance Status

ACTIVE VALIDATION — COMPILED RUNTIME NOT YET THE DEFAULT

Until the remaining structured-output guard is cleared:

292.59 tok/s remains the validated NVFP4 release throughput

while:

636.46 tok/s remains the current quality-preserving compiled candidate

The question is not simply:

Is it faster?

The question is:

Is it faster without making the model worse?

---

⚡ Performance Status

The external numbers are not quality-equivalent comparisons.

The current priority is:

push the quality-oriented configuration substantially faster before considering reductions to its denoising budget

---

🎮 Consumer Blackwell

E38 NVFP4 should also be a strong fit for the GeForce RTX 5090.

The 18.86 GB checkpoint fits within 32 GB VRAM, making consumer Blackwell a practical deployment target.

Similar single-model performance is expected.

Current measured E38 NVFP4 performance figures are from the RTX PRO 6000 Blackwell. Direct RTX 5090 validation is still needed.

---

📊 Aligned Validation Results

The major validation results are aligned across:

• Base BF16
• E38 BF16
• E38 NVFP4

using matched evaluation sets.

Aligned validation classification: PASS

E38 NVFP4 preserved the E38 behavior and capability profile under matched testing.

NVFP4 integrity validation confirmed:

• 12 / 12 artifact files matched
• all 20 E38-modified tensors remain exact BF16
• vision weights remain unchanged

---

🔵 What Is E38?

E38 is the selected configuration from a controlled Heretic-based abliteration search of:

google/diffusiongemma-26B-A4B-it

More than:

80 controlled candidate configurations

were evaluated before E38 was frozen.

The final E38 modification affects:

• Layers: 7–16
• Projections: attn.o_proj + mlp.down_proj
• Modified language tensors: 20
• Vision tensors modified: 0
• Precision: BF16

The checkpoint was frozen after selection.

The E38 modification remained unchanged through subsequent NVFP4 conversion and validation.

Frozen E38 Overlay SHA256

9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b

Frozen Selection Hash

cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175

No E38.1 checkpoint was created.

---

🔓 Refusal Behavior

The defining E38 behavioral change is its substantial reduction in target refusal behavior.

The aligned full refusal comparison used:

• 134 unique target-refusal prompts
• 83 unique benign / false-refusal prompts
• 217 unique refusal-related prompts
• 402 target generations per model
• 249 benign generations per model
• 651 refusal-related generations per model

The defining E38 refusal-suppression behavior was directly preserved after NVFP4 conversion.

E38 and E38 NVFP4 should therefore not be expected to retain the refusal behavior of the original upstream model.

---

🟢 Objective Quality

The frozen matched 200-prompt evaluation produced:

E38 NVFP4 versus E38 BF16:

• Difference: +1.5 percentage points
• 95% CI: -2.0 to +5.0 pp
• p = 0.5811

The difference was not statistically significant.

Validated conclusion: NVFP4 preserved measured E38 BF16 quality.

The higher NVFP4 point estimate is not claimed as proof that quantization makes the model more capable.

---

📈 Public Benchmark Results

The larger public benchmark suite was run on Base BF16 and E38 BF16.

The full benchmark suites have not all been rerun directly on NVFP4.

A separate aligned 100-item Base/E38/NVFP4 subset found:

• Base BF16: 62/100
• E38 BF16: 58/100
• E38 NVFP4: 64/100

The smaller aligned set is treated as a:

regression detector, not evidence of NVFP4 superiority

E38 is best described as a:

capability redistribution

rather than a universally stronger checkpoint.

---

🧮 MATH Level 5

The full matched MATH Level 5 evaluation used:

1,324 problems

at matched 1280-token generation capacity.

Difference:

-3.55 percentage points

with:

• 95% CI: -5.59 to -1.44 pp
• McNemar p: 0.00119

The mathematics regression is a real measured E38 BF16 tradeoff under the tested inference configuration.

The full MATH benchmark has not yet been rerun directly on NVFP4.

---

🖼️ Multimodal Evaluation

The aligned three-model multimodal subset produced:

A larger historical Base/E38 multimodal evaluation produced:

No aggregate multimodal capability loss was observed.

The E38 tensor audit independently confirmed:

0 vision tensors modified

---

✍️ Long-Form Language Quality

A prospective 432-generation total long-form study directly compared:

• Base BF16
• E38 BF16
• E38 NVFP4

The original Base model showed the same general class of rare grammar and lexical problems before either E38 abliteration or NVFP4 quantization.

The E38-vs-Base grammar difference was statistically unresolved:

• Difference: +1.420 errors /10k words
• 95% CI: -0.674 to +3.541
• McNemar p: 0.560

Therefore:

The long-form study did not establish an E38-specific grammar regression.

NVFP4 also did not show evidence of language-quality degradation.

The lower NVFP4 point estimates are encouraging but are not claimed as proof that quantization improves grammar.

---

🔬 DiffusionGemma May Have More Potential

The current working hypothesis is:

DiffusionGemma may possess greater language-quality capability than current diffusion inference is consistently extracting from the model.

This remains a research hypothesis.

Current evidence includes:

• the same general grammar / lexical error class appearing in Base, E38, and NVFP4
• approximately 3.01× higher error rate in the final 25% of a diffusion canvas
• approximately +0.0504 higher final entropy at error-prone positions
• approximately -0.0418 lower top-1 / top-2 confidence margin at error-prone positions
• simple additional denoising failing to establish a better overall quality tradeoff

These findings suggest that some language failures may be connected to diffusion finalization rather than being purely encoded in the learned weights.

If that is true:

additional language capability may already exist inside DiffusionGemma and could potentially be recovered through better inference

That could potentially improve:

• grammar
• lexical reliability
• long-form consistency
• structured generation
• difficult reasoning trajectories
• output stability
• general language quality

without necessarily requiring:

• more parameters
• a larger checkpoint
• full-model retraining

The next major research phase is intended to test this directly.

---

⚡ NVFP4 Deployment Performance

The NVFP4 derivative reduces checkpoint size from:

51.68 GB → 18.86 GB

The dedicated matched performance study measured:

Compared with E38 BF16, NVFP4 achieved:

• 63.5% smaller checkpoint
• 58.2% lower measured mean latency
• 75.2% higher measured throughput
• approximately 1.75× measured throughput

The current compiled quality-preserving candidate has subsequently reached:

636.46 tok/s overall

and:

586.63 tok/s long-form

on its matched A/B workload.

Those newer figures remain under validation.

View the NVFP4 Deployment Release

---

🧠 NVFP4 Memory-System Efficiency

A likely contributor to NVFP4's performance is improved memory-system efficiency.

The smaller routed-expert representation may reduce:

• expert-weight traffic
• data movement per expert execution
• memory-bandwidth pressure
• cache pressure

and may improve effective cache behavior.

This remains a:

working performance hypothesis

The large compiled-graph A/B improvement also shows that a substantial amount of previous runtime cost came from execution overhead rather than the frozen model weights themselves.

Current optimization is separating:

• memory traffic
• compute
• kernel-launch overhead
• graph execution
• cache behavior
• scheduling
• synchronization

---

⚙️ Recommended Quality-Oriented Inference Configuration

The retained quality-oriented configuration is:

• t_max = 0.80
• t_min = 0.40
• max_denoising_steps = 48
• entropy_bound = 0.1
• confidence_threshold = 0.005
• stability_threshold = 1
• adaptive stopping enabled
• canvas_length = 256
• long_form_capacity = 1280

This remains the:

validated quality target

The active NVFP4 performance work is attempting to execute this target substantially faster.

Compared with the published 16-step performance-only benchmark:

48 steps represents a 3× higher maximum denoising-step budget.

Adaptive stopping means this does not imply exactly 3× realized computation on every generation.

The goal remains:

maximum practical throughput with zero measurable quality regression

---

🧪 Testing Scope

The broader E38 research program includes:

• 80+ controlled abliteration candidates
• 3,300-generation sealed Base/E38 evaluation
• aligned full refusal testing
• 2,400-generation prospective inference study
• 1,324-problem MATH Level 5 evaluation
• 3,200-generation mathematics seed study
• 432-generation long-form language study
• 200-prompt matched Base/E38/NVFP4 validation
• full BBH, MuSR, MMLU-Pro, and IFEval
• multimodal testing
• multi-turn testing
• tensor-integrity auditing
• generation-capacity studies
• NVFP4 quality-retention testing
• dedicated latency / throughput testing
• diffusion-trajectory analysis
• active compiled / CUDA-graph throughput testing

The final aligned validation matched:

• 402 target-refusal generations per model
• 134 prompt-majority decisions per model
• 249 benign generations per model
• 100 benchmark-regression items per model
• 20 multimodal cases per model
• 8 multi-turn conversations per model
• 24 multi-turn generations per model

Negative findings and regressions are retained rather than discarded.

---

🧬 Model Lineage

Google DiffusionGemma 26B A4B BF16

↓

Heretic / controlled abliteration search

↓

E38 BF16 — this frozen validated reference

↓

E38 NVIDIA-style NVFP4 — validated deployment derivative

The BF16 checkpoint is intentionally retained so the effects of:

• abliteration
• quantization

can be evaluated independently.

---

🔐 Reproducibility

Upstream Base

google/diffusiongemma-26B-A4B-it

Pinned revision:

f7f5b7f5fa82ffc52addd066915886d497f5517b

E38 Integrity

Overlay SHA256:

9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b

Selection hash:

cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175

Measured E38 weight difference:

• Relative Frobenius difference: 0.0250308802854
• Maximum absolute weight difference: 0.201904296875

---

⚠️ Known Limitations

E38 is experimental and is not universally stronger than Base.

Known considerations include:

• substantially reduced refusal behavior
• weaker exact-format / strict-instruction performance
• lower matched MATH Level 5 performance
• occasional grammar and lexical errors also observed in Base DiffusionGemma
• stochastic diffusion inference
• sensitivity to output capacity and inference configuration

The current long-form evidence did not establish an E38-specific grammar regression.

The possibility that better diffusion inference could unlock additional language quality remains a:

research hypothesis

---

Intended Use

This checkpoint is intended for:

• diffusion-LM research
• abliteration research
• capability evaluation
• multimodal experimentation
• quantization reference
• diffusion-inference research
• local model experimentation
• reproducible comparison with the NVFP4 derivative

For deployment and current performance development:

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4

---

Safety and Behavior

This model intentionally has substantially reduced refusal behavior.

It may answer requests that the upstream DiffusionGemma checkpoint would refuse.

The aligned evaluation found:

Benign false refusal was:

0 / 249 for Base BF16, E38 BF16, and E38 NVFP4

This release should therefore not be interpreted as retaining the original model's refusal or safety characteristics.

Users should independently evaluate behavior appropriate for their application.

---

Upstream Model

Original upstream model:

google/diffusiongemma-26B-A4B-it

Recommended deployment derivative:

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4

---

License

Apache License 2.0.

This repository contains a modified derivative of:

google/diffusiongemma-26B-A4B-it