Downloads · 30 days
10
7% of all-time downloads
DSR1-1.5B-llmc-awq-w4
edge-inference/DSR1-1.5B-llmc-awq-w4
DSR1-1.5B-llmc-awq-w4 is a text generation model from edge-inference. Use it when you need the model to write or continue text. The card lists the license as bsd-3-clause.
AWQ quantized version of DeepSeek-R1-Distill-Qwen-1.5B using llm-compressor.
All-time downloads
134
Public
Parameters
1.8B
1.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 GB · 99%
Parameter types
How the weights are stored.
I321.3B · 73%
Try a prompt
Base models
Model card
From the Hugging Face model README
DSR1-1.5B-llmc-awq-w4
AWQ quantized version of DeepSeek-R1-Distill-Qwen-1.5B using llm-compressor.
Model Details
- Base Model: DeepSeek-R1-Distill-Qwen-1.5B
- Quantization: AWQ W4A16 (4-bit weights, 16-bit activations)
- Group Size: 128
- Framework: llm-compressor
- Memory: 1.6GB (vs 3GB original)
Usage
vLLM
from vllm import LLM
model = LLM("edge-inference/DSR1-1.5B-llmc-awq-w4")
output = model.generate("Hello, how are you?")
print(output[0].outputs[0].text)
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("edge-inference/DSR1-1.5B-llmc-awq-w4")
tokenizer = AutoTokenizer.from_pretrained("edge-inference/DSR1-1.5B-llmc-awq-w4")
Performance
- Memory: 47% reduction (1.6GB vs 3GB)
- Speed: Faster inference due to reduced memory bandwidth
- Quality: Minimal degradation with AWQ quantization
License
Same as base model (DeepSeek License)
- Task
- Text Generation
- Type
- qwen2
- License
- bsd-3-clause
- Languages
- en
- Created
- Aug 30, 2025
- Updated
- Aug 30, 2025