Downloads · 30 days
51
71% of all-time downloads
ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8
inference-optimization/ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8
ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8 is a machine learning model from inference-optimization. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Test checkpoint — not a production release. Trained to validate the FP8-quantized hidden-states transfer backend added in vllm-project/speculators1028 (draft, reviving 491).
All-time downloads
72
Public
Parameters
1.8B
3.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.5 GB · 100%
Parameter types
How the weights are stored.
BF161.8B · 100%
Model card
From the Hugging Face model README
ctest: Qwen3-8B DFlash speculator — FP8 hidden-states ablation (fp8)
Test checkpoint — not a production release. Trained to validate the FP8-quantized hidden-states transfer backend added in vllm-project/speculators#1028 (draft, reviving #491).
- Verifier:
Qwen/Qwen3-8B - Algorithm: dflash
- Hidden-states precision during training: fp8 (
new FP8 backend, per-token float8_e4m3fn quantization) - Training data: 5K magpie + 5K ultrachat samples from
inference-optimization/Dataset-Qwen3-235B-Instruct
This is one of a matched pair (...-fp8ablation-bf16 / ...-fp8ablation-fp8) trained
identically except for the hidden-states transfer precision, to isolate the effect of
FP8 quantization on speculator quality. See the sibling repo for the other precision.
Validation metrics (final epoch)
loss_epoch: 0.669430full_acc_epoch: 0.190271position_1_acc_epoch: 0.672740position_2_acc_epoch: 0.472040position_3_acc_epoch: 0.345522position_4_acc_epoch: 0.260192position_5_acc_epoch: 0.199753position_6_acc_epoch: 0.156152position_7_acc_epoch: 0.124998position_8_acc_epoch: 0.103757position_9_acc_epoch: 0.089780position_10_acc_epoch: 0.080512position_11_acc_epoch: 0.074648position_12_acc_epoch: 0.071026position_13_acc_epoch: 0.067589position_14_acc_epoch: 0.064986position_15_acc_epoch: 0.062941eal_epoch: 1.169300
guidellm serving eval
See acceptance.csv in this repo for the full per-subset guidellm breakdown (9 subset rows).
Full ablation writeup, all 6 checkpoints' results, exact commands, and code: see the results package referenced from PR #1028.
- Created
- Aug 21, 2026
- Updated
- Aug 21, 2026