Downloads · 30 days
16
2% of all-time downloads
huluhuluu/qwen3-1.7b-eagle3-k2-sw256-sharegpt-epoch2-step65000
qwen3-1.7b-eagle3-k2-sw256-sharegpt-epoch2-step65000 is a text generation model from huluhuluu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B. It is a draft model for speculative decoding, not a standalone target language model.
Downloads · 30 days
16
2% of all-time downloads
All-time downloads
879
Public
Parameters
196M
392 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors392 MB · 100%
How the weights are stored.
BF16196M · 100%
From the Hugging Face model README
This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.
| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-k2-sw256-sharegpt |
| Checkpoint | epoch_2_step_65000 |
| Epoch | 2 |
| Global step | 65000 |
| Files | config.json, model.safetensors, training_state.pt (when present) |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training |
| Training data | sharegpt_train.jsonl |
| Learning rate | 0.0001 |
| Batch size | 1 |
| Target batch size | 1 |
| Epochs configured | 10 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 5 |
| Draft layers | 2 |
| Draft sliding window | 256 |
| Save interval | 5000 |
| Eval interval | 5000 |
| Seed | 0 |
| TP / DP size | 1 / 4 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 64 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 2 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Sliding window | 256 |
| Max window layers | 2 |
| Future hidden | Not recorded |
training_state.pt is included when available for provenance and training-state inspection.