Downloads · 30 days
19
42% of all-time downloads
huluhuluu/qwen3-1.7b-eagle3-sharegpt-epoch9-step233900
qwen3-1.7b-eagle3-sharegpt-epoch9-step233900 is a text generation model from huluhuluu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B. It is a draft model for speculative decoding, not a standalone target language model.
Downloads · 30 days
19
42% of all-time downloads
All-time downloads
45
Public
Parameters
137M
274 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors274 MB · 100%
How the weights are stored.
BF16137M · 100%
From the Hugging Face model README
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.
| Field | Value |
|---|---|
| Source run | qwen3-1p7b-eagle3-sharegpt |
| Checkpoint | epoch_9_step_233900 |
| Epoch | 9 |
| Global step | 233900 |
| Files | config.json, model.safetensors, training_state.pt (when present) |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training |
| Training data | sharegpt_train.jsonl |
| Learning rate | 0.0001 |
| Batch size | 1 |
| Target batch size | 1 |
| Epochs configured | 10 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 7 |
| Draft layers | Not recorded |
| Draft sliding window | Not recorded |
| Save interval | 5000 |
| Eval interval | 5000 |
| Seed | 0 |
| TP / DP size | 1 / 4 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 64 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 1 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Sliding window | Not recorded |
| Max window layers | Not recorded |
| Future hidden | Not recorded |
training_state.pt is included when available for provenance and training-state inspection.