Downloads · 30 days
5
2% of all-time downloads
Llama-3.2-1B-vknn
katolikov/Llama-3.2-1B-vknn
Llama-3.2-1B-vknn is a text generation model from katolikov. Use it when you need the model to write or continue text. It is set up for vknn. The card lists the license as llama3.2.
Built with Llama. Llama-3.2-1B-Instruct compiled to VKNN's .vxm format for on-device GPU inference via the Vulkan backend.
All-time downloads
328
Public
Repo size
1.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.vxm1.2 GB · 99%
At a glance
- Task
- Text Generation
- Library
- vknn
- License
- llama3.2
- Model type
- llama
- Access
- Public
- Created
- Jul 11, 2026
- Updated
- Jul 11, 2026
- SHA
- b252f8f1
Try a prompt
Base models
Model card
From the Hugging Face model README
Llama-3.2-1B-Instruct — VKNN .vxm (GPU, on-device)
Built with Llama. Llama-3.2-1B-Instruct compiled to VKNN's .vxm format for on-device GPU inference via the Vulkan backend.
| file | what |
|---|---|
llama-3.2-1b-instruct-int4.vxm | Instruct, int4 weights (-Os: calibration-free weight quantization + AWQ outlier columns), C = 1024 + a 256-token prefill bucket (~1.2 GB), 100% Vulkan (0 CPU fallback). GQA: 16 layers, 8 KV heads, head_dim 64, vocab 128256. |
tokenizer.json / tokenizer_config.json / config.json | the Llama-3 byte-level BPE tokenizer + config |
The two-bucket plan ingests the prompt in one 256-token prefill forward, then decodes with the engine-resident KV cache. vknn_chat selects the buckets automatically; stop on <|eot_id|> (id 128009). Verified on device: greedy decode stays coherent with 0 CPU fallback across both buckets (675-node prefill + 354-node decode, all Vulkan).
This is a GPU build — every tensor-compute op runs on the Vulkan backend — an int4 weight-only requantization of the fp16 export.
License
Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Use of this derivative is subject to that license and Meta's Acceptable Use Policy.
- Task
- Text Generation
- Library
- vknn
- Type
- llama
- License
- llama3.2
- Created
- Jul 11, 2026
- Updated
- Jul 11, 2026