Downloads · 30 days
7
15% of all-time downloads
aya-expanse-8b-compressed-final
AnishRacherla/aya-expanse-8b-compressed-final
aya-expanse-8b-compressed-final is a translation model from AnishRacherla. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
Fully self-contained standalone model. No other repo required.
All-time downloads
46
Public
Parameters
7.3B
5.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.2 GB · 100%
Parameter types
How the weights are stored.
U86.3B · 86%
Try a prompt
Model card
From the Hugging Face model README
Aya Expanse 8B — Compressed Final (Pruned + Fine-tuned + int4)
Fully self-contained standalone model. No other repo required.
Compression Pipeline
| Step | Method | Detail |
|---|---|---|
| 1 | Layer Pruning | 32 → 28 layers, 16 GB → 14 GB |
| 2 | LoRA Fine-tuning | QLoRA on WMT19 En-Zh (36k samples) |
| 3 | Weight Merging | merge_and_unload() — no PEFT at runtime |
| 4 | int4 Quantization | NF4 double-quant via bitsandbytes |
How to Load
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type='nf4', bnb_4bit_compute_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained('AnishRacherla/aya-expanse-8b-compressed-final', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('AnishRacherla/aya-expanse-8b-compressed-final', quantization_config=bnb, device_map='auto', trust_remote_code=True)
Benchmark (FLORES-200, 100 samples, En→Zh)
| Metric | fp16 (merged) | int4 — this model | Δ |
|---|---|---|---|
| VRAM | 13.34 GB | 4.89 GB | -8.46 GB |
| Load Time | 101.5s | 47.2s | -54.3s |
| Tokens/sec | 14.9 | 10.7 | -4.2 |
| COMET Score | 0.8642 | 0.8553 | -0.0089 |
- Task
- Translation
- Type
- cohere
- License
- apache-2.0
- Languages
- en, zh
- Created
- Jun 27, 2026
- Updated
- Jun 27, 2026