Downloads · 30 days
17
4% of all-time downloads
huluhuluu/qwen3-1.7b-eagle3-sharegpt-sw768-epoch9-step455000
qwen3-1.7b-eagle3-sharegpt-sw768-epoch9-step455000 is a text generation model from huluhuluu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B. It is a draft model for speculative decoding, not a standalone target language model.
Downloads · 30 days
17
4% of all-time downloads
All-time downloads
407
Public
Parameters
137M
274 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors274 MB · 100%
How the weights are stored.
BF16137M · 100%
From the Hugging Face model README
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.
| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-sharegpt-sw768 |
| Checkpoint | epoch_9_step_455000 |
| Epoch | 9 |
| Global step | 455000 |
| Files | config.json, model.safetensors, training_state.pt |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training |
| Framework revision | 9fbbde8ab5d6ee69fb0af3701330027b8beca37a |
| Training data | sharegpt_train.jsonl |
| Learning rate | 0.0001 |
| Batch size | 1 |
| Epochs configured | 10 |
| Total scheduled steps | 467800 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 7 |
| Draft accumulation steps | 1 |
| Draft sliding window | 768 |
| Save / eval interval | 5000 / 5000 |
| Seed | 0 |
| TP / DP size | 1 / 2 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 16 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 1 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Sliding window | 768 |
| Max window layers | 28 |
training_state.pt is included for provenance and training-state inspection.