Downloads · 30 days
21
16% of all-time downloads
taide-eagle3-vllm
seanmamasde/taide-eagle3-vllm
taide-eagle3-vllm is a machine learning model from seanmamasde. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
The license is inherited from the TAIDE Model.
All-time downloads
133
Public
Repo size
1.8 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin907 MB · 100%
At a glance
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Jul 23, 2025
- Updated
- Jul 24, 2025
- SHA
- fa463042
Model card
From the Hugging Face model README
The license is inherited from the TAIDE Model.
This is an Eagle3 model for Llama-3.1-TAIDE-LX-8B-Chat, trained on custom sharegpt_gpt4 dataset, and for inferencing using vllm.
Following benchmark was ran with this benchmarking file and these settings:
-
A single H100 GPU
-
dtype:float16 -
attention-backend:flashinfer -
speculative_config:method:eagle3draft_tensor_parallel_size:1num_speculative_tokens:2
-
num_prompts:1 -
Lhs: Baseline
-
Rhs: Eagle3
-
Achieving around 1.32x bump in inferencing speed
- Type
- llama
- License
- other
- Created
- Jul 23, 2025
- Updated
- Jul 24, 2025