Downloads · 30 days
204
3% of all-time downloads
squanchyzx/DeepSeek-V4-Flash-0731-HERETIC-Abliterated-FP8
DeepSeek-V4-Flash-0731-HERETIC-Abliterated-FP8 is a text generation model from squanchyzx. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
A native-FP8 behavioral derivative of deepseek-ai/DeepSeek-V4-Flash-0731.
Downloads · 30 days
204
3% of all-time downloads
All-time downloads
6.5K
Public
Parameters
305B
169 GB on disk
Likes
21
Public
Click a slice to open those files.
.safetensors168 GB · 100%
How the weights are stored.
I8296B · 97%
From the Hugging Face model README
A native-FP8 behavioral derivative of deepseek-ai/DeepSeek-V4-Flash-0731.
v2 replaces the original λ1.5 release on main. The original commit remains available as an immutable tagged revision because it developed a long-context degeneration failure in real agent histories. Do not deploy the old revision for long-lived agent sessions.
The issue was not a single repeated word. In long tool/agent histories, v1 could fall into several degeneration patterns:
kanka, my, or similar tokens;() fragments or repeated punctuation;The failure came from an overly aggressive rank-3 edit at lambda=1.5, combined with speculative/MTP amplification. Merely restoring stock MTP was not enough: the λ1.5 stock-MTP control still produced a 324-character consecutive 我 run.
chat, think-high, and think-max.10–42.lambda=1.5 to lambda=1.35.This checkpoint was produced with Heretic v1.4.0 by Philipp Emanuel Weidmann and contributors, using base commit 7675b90d648154cdfefa597372cc477df9848eab. Heretic is their project and is licensed AGPL-3.0-or-later; it is not owned or authored by this model publisher.
The release-specific work here is the DeepSeek V4/mHC two-node adaptation, separate refusal-direction capture, rank-3 subspace construction, native-FP8 attention-only bake, long-context incident reproduction, generic degeneration detector, and release evaluation. See HERETIC_ATTRIBUTION.md.
66 (33 weight + scale pairs)72,25101,107,370,672ca4a043ae3a306a50b680a08c3f175d7a7df20b955f28a9d4e45161692c45638393a1b09ffacdf5e4e226a84f0b8d5467947ad0f35c91a71127e609003375a560/48 on both nodesThe detector does not key on one literal word. It evaluates answer and reasoning content for same-word/character/symbol runs, repeated 2/3/4/8-grams, repeated lines, dominant-token fraction, vocabulary uniqueness, entropy, compression, empty-parenthesis floods, and unexpected script switching.
Results with speculative decoding enabled:
92,501 prompt tokens: 5/5 seeds clean64,245 prompt tokens: 5/5 seeds clean124,983 prompt-token replay: clean249,971 prompt-token replay: clean12/12 clean0/126/6 valid and correct11/126/16 (37.5%; lower is less refusal)These are bounded release tests, not a universal capability or safety benchmark. Full compact receipts are in EVAL_RESULTS.md and the eval/v2/ directory.
Use a DeepSeek-V4-compatible runtime and the official encoder. This checkpoint retains the official V4/DSpark architecture and native FP8 layout. Generic runtimes without DeepSeek V4 support may not load it correctly.
The release candidate was exercised on two NVIDIA DGX Spark nodes with tensor parallelism 2, a 1,048,576-token configured context window, the official V4 encoder, and a DeepSeek-V4-capable vLLM build.
No checkpoint revision was created for this update. The v2 weight shards, index, configuration, tokenizer, and immutable v2-lam1p35-mtpstock tag are unchanged. This section documents a verified runtime profile and a bounded matched comparison against the unmodified official DeepSeek-V4-Flash-0731 checkpoint.
The qualified two-DGX-Spark deployment uses the runtime lineage from MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark at commit 0817b11f5bfd53aeb85c90d1e8ae3755898b5314. That repository supplies runtime/deployment fixes; it is not a separate Mia-tuned checkpoint.
The validated profile kept this repository's v2 weights and used TP=2, MTP=3, a 1,048,576-token configured window, the official encoder, regular CUDA graphs, and the NVFP4 sparse-MLA fix. Direct API gates passed 5/5, including strict JSON and tool-history re-encoding. Long-context gates completed at 66,637, 249,971, and 609,847 prompt tokens without OOM, engine death, traceback, HTTP 5xx, or container restart. The 609,847-token run decoded at 38.768 tok/s, but its prefill/TTFT was 541.203 s; the configured 1M window should not be presented as low-latency interactive UX.
The qualified deployment currently uses host-mounted patch files. An immutable runtime image and mount-independence receipt remain future work.
Both arms were cold-loaded into the same TP=2 runtime envelope with identical serving flags, tokenizer, prompts, seeds, sampling, MTP=3, and default thinking=false chat-template behavior.
14/20; Heretic v2 11/20.3; Heretic-only wins: 0; exact paired McNemar two-sided p=0.25.6/6.11/12.10/12; Heretic v2 9/12.15/16; Heretic v2 6/16 (lower means less refusal).6, Heretic v2 2, ties 4; mean scores stock 7.500, Heretic v2 7.417.Interpretation: stock showed a directional edge on this small strict objective suite, while v2 preserved much lower refusal behavior and equal tool-call compliance. The sample is too small to establish a universal capability gap. Use the official stock checkpoint when upstream-aligned behavior and maximum baseline correctness are the priority; evaluate this derivative when reduced refusal behavior is an explicit requirement. Do not treat abliteration as a free quality improvement.
Machine-readable receipts: matched-ab-summary.json, blind-judge-summary.json, suite-manifest.json, and runtime-qualification.json.
main: current v2 releasev2-lam1p35-mtpstock: immutable v2 tagv1-lam1p5-known-long-context-issue: preserved v1 tag; not recommended for long-lived agent sessionsThe upstream repository and model weights are MIT licensed. This derivative retains the MIT license and original DeepSeek copyright notice.
Base model: deepseek-ai/DeepSeek-V4-Flash-0731