Downloads · 30 days
16
33% of all-time downloads
taide-eagle3-sglang
seanmamasde/taide-eagle3-sglang
taide-eagle3-sglang is a machine learning model from seanmamasde. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
The license is inherited from the TAIDE Model.
All-time downloads
49
Public
Repo size
1.8 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin907 MB · 100%
At a glance
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Jul 23, 2025
- Updated
- Jul 24, 2025
- SHA
- ad612c06
Model card
From the Hugging Face model README
The license is inherited from the TAIDE Model.
This is an Eagle3 model for Llama-3.1-TAIDE-LX-8B-Chat, trained on custom sharegpt_gpt4 dataset, and for inferencing using sglang.
Following benchmark was ran with this benchmarking file and these settings:
-
A single H100 GPU
-
dtype:float16 -
attention-backend:flashinfer -
mem-fraction-static:0.8 -
max-total-tokens:131072 -
cuda-graph-max-bs:32 -
speculative decoding related:
speculative-algorithm:EAGLE3speculative-num-steps:3speculative-eagle-topk:24speculative-num-draft-tokens:128
-
num_prompts:1 -
Lhs: Baseline
-
Rhs: Eagle3
-
Achieving around 1.56x bump in inferencing speed
- Type
- llama
- License
- other
- Created
- Jul 23, 2025
- Updated
- Jul 24, 2025