Downloads · 30 days
13
18% of all-time downloads
aya-expanse-8b-compressed-final-int4
AnishRacherla/aya-expanse-8b-compressed-final-int4
aya-expanse-8b-compressed-final-int4 is a translation model from AnishRacherla. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
All-time downloads
71
Public
Parameters
7.3B
5.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.2 GB · 100%
Parameter types
How the weights are stored.
U86.3B · 86%
Try a prompt
Model card
From the Hugging Face model README
Aya Expanse 8B — Compressed (int4)
Self-contained. No other repo needed.
Pipeline: Pruning → LoRA fine-tune → Merge → int4 quantize
Load
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16)
tok = AutoTokenizer.from_pretrained('AnishRacherla/aya-expanse-8b-compressed-final-int4', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('AnishRacherla/aya-expanse-8b-compressed-final-int4', quantization_config=bnb, device_map='auto', trust_remote_code=True)
Benchmark vs Baseline (FLORES-200, 100 En→Zh)
| Metric | Baseline int4 | This model | Δ |
|---|---|---|---|
| VRAM | 5.31 GB | 4.89 GB | -0.42 |
| Tokens/sec | 9.2 | 9.8 | +0.6 |
| COMET | 0.5095 | 0.8553 | +0.3458 |
- Task
- Translation
- Type
- cohere
- License
- apache-2.0
- Languages
- en, zh
- Created
- Jun 29, 2026
- Updated
- Jun 29, 2026