Downloads · 30 days
32
10% of all-time downloads
Qwen3-0.6B-nanoquant-GGUF
arelath/Qwen3-0.6B-nanoquant-GGUF
Qwen3-0.6B-nanoquant-GGUF is a text generation model from arelath. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
- Status: completed - Model: Qwen/Qwen3-0.6B - Revision: c1899de289a04d12100db370d81485cdf75e47ca - Candidate run: D:\dev\research\NanoQuantRewrite\evidence\028\028-compress-and-benchmark-qwen3-0-6b - Backend: llama.c…
All-time downloads
319
Public
Repo size
393 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf393 MB · 99%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026
- SHA
- 3eade90c
Try a prompt
Base models
Model card
From the Hugging Face model README
Experiment 28: Qwen/Qwen3-0.6B quality benchmark
- Status:
completed - Model:
Qwen/Qwen3-0.6B - Revision:
c1899de289a04d12100db370d81485cdf75e47ca - Candidate run:
D:\dev\research\NanoQuantRewrite\evidence\028\028-compress-and-benchmark-qwen3-0-6b - Backend:
llama.cpp - Wall time: 336.10 seconds
completed means all evaluators returned finite metrics; it is not a BF16-quality acceptance gate.
Protocol
- WikiText-2: 64 samples × 128 tokens, batch 8
- WikiText token hash:
sha256:e85e3ae8da01294229ac7dfc651f5d44f74eadaff974ad80a5a19779b58fbe0f - Tasks: piqa, arc_easy, arc_challenge, hellaswag, winogrande, boolq; first 200 rows, batch 4
- Tokenizer hash:
sha256:af4e928cfa630ef78563f4800a751d568307a7cd4d4f5fd5919de27b0fbfdbc4
Quality results
| Benchmark | Metric | BF16 | NanoQuant | Delta | Ratio |
|---|---|---|---|---|---|
| WikiText-2 | perplexity ↓ | 55.166982 | 292.263889 | +237.096907 (+429.78%) | 5.2978x |
| piqa | acc_norm ↑ | 0.6900 | 0.5500 | -0.1400 | 0.7971x |
| arc_easy | acc_norm ↑ | 0.5650 | 0.3450 | -0.2200 | 0.6106x |
| arc_challenge | acc_norm ↑ | 0.3400 | 0.2600 | -0.0800 | 0.7647x |
| hellaswag | acc_norm ↑ | 0.5100 | 0.3950 | -0.1150 | 0.7745x |
| winogrande | acc ↑ | 0.6050 | 0.5150 | -0.0900 | 0.8512x |
| boolq | acc ↑ | 0.6550 | 0.6250 | -0.0300 | 0.9542x |
Deployment storage
| Deployable representation | Bytes | Storage savings vs BF16 |
|---|---|---|
| BF16 checkpoint tensors | 1,503,300,328 | — |
| NanoQuant GGUF | 393,472,384 | 73.83% |
The packed NanoQuant quantized-layer payload is 56,725,568 bytes. The complete GGUF is the comparable deployable artifact and also includes embeddings, the output projection, norms, and metadata.
Quality execution
The BF16 reference uses Transformers. The NanoQuant candidate is evaluated only from the exported GGUF by the pinned llama.cpp runtime; no reconstructed dense or factorized PyTorch candidate is loaded.
| Runtime | Elapsed seconds | Peak dedicated GPU bytes | Peak host bytes |
|---|---|---|---|
| BF16 Transformers | 28.94 | 5,205,131,264 | 2,352,574,464 |
| NanoQuant llama.cpp GGUF | 306.33 | 1,096,601,600 | 997,515,264 |
- GGUF:
D:\dev\research\NanoQuantRewrite\Results\028\qwen3-0-6b-nanoquant.gguf - GGUF SHA-256:
sha256:b0ffbcc384324f5500a040c78fc649197925fdccf30139b9335cdda8b3cf6011 - llama.cpp commit:
2927569f318311aded148d76be6b1109664066f5
Provenance
- Experiment config hash:
sha256:8a0ebce1b4e0ccbaeef076b5a4cd0910e9f137984f66f2a39b47642416daeeda - Launcher:
experiments/028-compress-and-benchmark-qwen3-0-6b.py - Candidate identity:
{"config_hash":"sha256:6f5ef817ff06cfa29ee046f08800b7179bb574efc0ad914dce1a717e8d94fa29","model_hash":"sha256:be8e62ef3fcac5fb56dc193b416d77ca862ec1594958ada22bd5c002f6a065de","plan_hash":"sha256-3fe3842dd0ccde5b34961fd71302e681ecadf3b0653d9e9f49131aadfe0f2d51"} - Global tuning:
None
- Task
- Text Generation
- License
- apache-2.0
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026