Downloads · 30 days
36
4% of all-time downloads
huluhuluu/qwen3-1.7b-eagle3-k2-sw256-mha-sharegpt-epoch9-step171560
qwen3-1.7b-eagle3-k2-sw256-mha-sharegpt-epoch9-step171560 is a text generation model from huluhuluu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B. It is a draft model for speculative decoding, not a standalone target language model.
Downloads · 30 days
36
4% of all-time downloads
All-time downloads
916
Public
Parameters
213M
425 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors425 MB · 100%
How the weights are stored.
BF16212M · 100%
From the Hugging Face model README
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.
| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-k2-sw256-mha-sharegpt |
| Checkpoint | epoch_9_step_171560 |
| Epoch | 9 |
| Global step | 171560 |
| Files | config.json, model.safetensors, training_state.pt (when present) |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training |
| Training data | sharegpt_train.jsonl |
| Learning rate | 0.0001 |
| Batch size | 1 |
| Target batch size | 1 |
| Epochs configured | 10 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 5 |
| Draft layers | 2 |
| Draft sliding window | 256 |
| Save interval | 5000 |
| Eval interval | 5000 |
| Seed | 0 |
| TP / DP size | 1 / 4 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 64 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 2 |
| Attention heads | 16 |
| KV heads | 16 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Sliding window | 256 |
| Max window layers | 2 |
| Future hidden | Not recorded |
training_state.pt is included when available for provenance and training-state inspection.