Downloads · 30 days
7
11% of all-time downloads
vllm-medusa-vicuna-13b-v1.3
seliny2/vllm-medusa-vicuna-13b-v1.3
vllm-medusa-vicuna-13b-v1.3 is a machine learning model from seliny2. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
<div align="center"<img src="https://github.com/FasterDecoding/Medusa/blob/main/assets/logo.png?raw=true" alt="Medusa" width="100" align="center"</div <div align="center"<h1 Medusa: Simple Framework for Accelerat…
All-time downloads
61
Public
Parameters
819M
1.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 GB · 100%
At a glance
- Model type
- medusa
- Access
- Public
- Created
- Dec 9, 2025
- Updated
- Dec 9, 2025
- SHA
- 3cff8964
Model card
From the Hugging Face model README
<div align="center"><img src="https://github.com/FasterDecoding/Medusa/blob/main/assets/logo.png?raw=true" alt="Medusa" width="100" align="center"></div>
<div align="center"><h1> Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads</h1></div>
<p align="center">
| <a href="https://sites.google.com/view/
medusa-llm"><b>Blog</b></a> | <a href="https://github.com/FasterDecoding/Medusa"><b>Codebase</b></a> |
</p>
Installation
Method 1: With pip
pip install medusa-llm
Method 2: From source
git clone https://github.com/FasterDecoding/Medusa.git
cd Medusa
pip install -e .
Model Weights
| Size | Chat Command | Hugging Face Repo |
|---|---|---|
| 7B | python -m medusa.inference.cli --model FasterDecoding/medusa-vicuna-7b-v1.3 | FasterDecoding/medusa-vicuna-33b-v1.3 |
| 13B | python -m medusa.inference.cli --model FasterDecoding/medusa-vicuna-13b-v1.3 | FasterDecoding/medusa-vicuna-13b-v1.3 |
| 33B | python -m medusa.inference.cli --model FasterDecoding/medusa-vicuna-33b-v1.3 | FasterDecoding/medusa-vicuna-33b-v1.3 |
Inference
We currently support inference in the single GPU and batch size 1 setting, which is the most common setup for local model hosting. We are actively working to extend Medusa's capabilities by integrating it into other inference frameworks, please don't hesitate to reach out if you are interested in contributing to this effort.
You can use the following command for lauching a CLI interface:
python -m medusa.inference.cli --model [path of medusa model]
You can also pass --load-in-8bit or --load-in-4bit to load the base model in quantized format.
- Type
- medusa
- Created
- Dec 9, 2025
- Updated
- Dec 9, 2025