Downloads · 30 days
16
5% of all-time downloads
huluhuluu/qwen3-1.7b-eagle3-stage2-adapter-w4-epoch7-step187120
qwen3-1.7b-eagle3-stage2-adapter-w4-epoch7-step187120 is a text generation model from huluhuluu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This repository contains one SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B. It is a draft model for speculative decoding, not a standalone target language model.
Downloads · 30 days
16
5% of all-time downloads
All-time downloads
327
Public
Parameters
139M
277 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors277 MB · 100%
How the weights are stored.
BF16138M · 100%
From the Hugging Face model README
This repository contains one SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.
| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-stage2-adapter-w4 |
| Checkpoint | epoch_7_step_187120 |
| Epoch | 7 |
| Global step | 187120 |
| Files | config.json, model.safetensors, training_state.pt |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training with stage2 adapter |
| Dataset | ShareGPT-format SpecForge training set |
| Learning rate | 1e-4 |
| Batch size | 1 |
| Target batch size | 1 |
| Epochs configured | 8 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 7 |
| Save interval | 5000 |
| Eval interval | 5000 |
| Seed | 0 |
| TP / DP size | 1 / 4 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 64 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 1 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Stage2 adapter | enabled |
| Stage2 adapter bottleneck | 256 |
| Stage2 max depth | 7 |
| Stage2 depth power | 2.0 |
training_state.pt is included for provenance and training-state inspection.