Downloads · 30 days
109
24% of all-time downloads
onewhosighs/Apathy-Qwen3.8-27B-DFlash-drafter-v3
Apathy-Qwen3.8-27B-DFlash-drafter-v3 is a machine learning model from onewhosighs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A six-layer DFlash-family draft model for speculative decoding with Qwen3.8-27B. This repository contains BF16 source weights; Atlas quantizes the dense draft projections to NVFP4 when loading the speed profile.
Downloads · 30 days
109
24% of all-time downloads
All-time downloads
463
Public
Parameters
2.1B
4.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors4.3 GB · 100%
From the Hugging Face model README
A six-layer DFlash-family draft model for speculative decoding with Qwen3.8-27B. This repository contains BF16 source weights; Atlas quantizes the dense draft projections to NVFP4 when loading the speed profile.
This is a draft model, not a standalone language model. A target model must verify every proposed token.
On one DGX Spark / GB10, the answering optimized-qwen target and this drafter
measured the following on Weschera's fixed MinHeap prompt, single stream,
temperature 0, reasoning disabled, 400 output tokens:
| Profile | Draft vocabulary | Repetitions | Median server decode tok/s |
|---|---|---|---|
| BF16 target KV | 96,000 | 5 | 61.7526 |
| NVFP4 target KV | 96,000 | 5 | 74.0882 |
| NVFP4 target KV | 248,320 (full) | 5 | 72.1689 |
Full-vocabulary rates were 70.6411, 72.1877, 72.2574, 72.0858, and
72.1689 tok/s. All responses had stable output SHA-256
f51d8358ea2a5c63353ca00a29208ae2cccd3039b070043cad514cc4af9761c4.
The complete result is included at
benchmarks/weschera-minheap-400tok-fullvocab-nvfp4kv-5x.json.
A post-publication end-to-end repeatability gate then ran the same request ten
more times against the unchanged server. All ten responses retained the exact
stable output hash above. Decode throughput had median 72.321 tok/s, mean
72.151 tok/s, standard deviation 0.712 tok/s, and 0.987% coefficient of
variation (range 70.799-72.929 tok/s). Thus the token output is deterministic
for this probe, while throughput is repeatable within normal runtime variance,
not numerically deterministic. The result is included at
benchmarks/weschera-minheap-400tok-fullvocab-nvfp4kv-e2e-10x.json.
NVFP4 target KV changed the target output relative to BF16 and FP8 KV. The 72.1689 figure is therefore a deterministic throughput result for this fixed probe, not evidence that NVFP4 KV is output-equivalent or broadly quality-equivalent to BF16. Every response reached the 400-token cap, so the probe also does not certify task completion. Use BF16 KV for reference behavior and separately qualify NVFP4 KV on your workloads.
| Field | Value |
|---|---|
| Architecture | DFlashDraftModel |
| Layers | 6 |
| Hidden size | 5,120 |
| Vocabulary | 248,320 |
| Block size | 16 |
| Maximum draft width | gamma 15 |
| Target capture layers | 1, 10, 18, 27, 35, 44, 52, 61 |
| Tensors | 69 |
| Weights | BF16 safetensors, 4,257,372,464 bytes |
model.safetensors SHA-256 | c15685d680bd58939689dcb4c344bb325efb75536df16d77c38517ce3df2dd6c |
config.json SHA-256 | c24cb2ef1c9788f90ed05870e9df70524f75ea84ceb4809909e1afcaaee22882 |
dflash.py SHA-256 | 1073c6dd9c71b404920739f183cd8ff431b82748f6210abaca6cbc7bfc11112f |
block_size=16 means trained_drafts = block_size - 1; use gamma 15. Atlas
refuses widths above the checkpoint contract.
The included serve-atlas-gb10-72.sh records the measured single-stream
profile without requiring changes to the Atlas Git repository:
MODEL_DIR=/path/to/optimized-qwen \
DRAFT=/path/to/this-repository \
SPARK_BIN=/path/to/apathy-atlas/target/release/spark \
./serve-atlas-gb10-72.sh
Then run the fixed five-repeat harness from the Atlas performance branch:
python3 bench/qwen38-gb10/weschera_minheap_repro.py \
--endpoint http://127.0.0.1:8896/v1/chat/completions \
--output /tmp/weschera.json \
--repetitions 5 --max-tokens 400 --store-output
The measured binary SHA-256 was
ebe7a7c8408bcd2d2aa492273aba8bc608c625e3e4c6c22831d014824c44f7d5.
Performance is workload-, engine-, binary-, and hardware-specific.
The exact three-file runtime export was retained without its optimizer state, training-state sidecar, corpus manifest, or immutable training manifest. This publication therefore does not claim an exact reproducible training lineage. Neighboring campaign notes are not substituted for missing artifact provenance.
What is verified is the published runtime artifact itself: file hashes, tensor count/shape contract, successful Atlas loading, deterministic target-verified decoding, acceptance telemetry, and the benchmark JSON above. The base target family is Apache-2.0 Qwen3.8-27B.
Do not infer broad quality, safety, long-context, or training-corpus claims from the MinHeap throughput result.