Downloads · 30 days
8
14% of all-time downloads
sarvam-1-int4-awq
D3vil-Reap3r/sarvam-1-int4-awq
sarvam-1-int4-awq is a machine learning model from D3vil-Reap3r. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a quantized version of the sarvamai/sarvam-1 2B parameter model. It has been optimized for high-performance inference on memory-constrained edge hardware.
All-time downloads
58
Public
Repo size
1.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.8 GB · 99%
At a glance
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Mar 1, 2026
- Updated
- Mar 1, 2026
- SHA
- 3829730f
Model card
From the Hugging Face model README
Sarvam-1 INT4-AWQ for TensorRT-LLM
Model Description
This is a quantized version of the sarvamai/sarvam-1 2B parameter model. It has been optimized for high-performance inference on memory-constrained edge hardware.
Quantization Details
- Method: W4A16 AWQ (Activation-aware Weight Quantization)
- Tool: NVIDIA ModelOpt
- Precision: INT4 Weights, FP16 Activations
Performance Benchmarks (NVIDIA RTX 3050 4GB)
Using the TensorRT-LLM C++ Executor API:
- Throughput: 86.77 tokens/sec
- TTFT: 22.75 ms
- VRAM Footprint: 3.03 GB (Strictly Fenced)
Usage
This model can be compiled into a TensorRT engine using trtllm-build.
trtllm-build --checkpoint_dir . \
--output_dir ./engine \
--gemm_plugin float16 \
--max_batch_size 1 \
--max_input_len 512 \
--max_seq_len 1024
Disclaimer
This model is a derivative of the Sarvam-1 model. Please refer to the original license for usage restrictions.
- Type
- llama
- License
- apache-2.0
- Created
- Mar 1, 2026
- Updated
- Mar 1, 2026