Downloads · 30 days
5
16% of all-time downloads
caeleste-speech-qwen2.5-0.5b-instruct-nf4
bschooled/caeleste-speech-qwen2.5-0.5b-instruct-nf4
caeleste-speech-qwen2.5-0.5b-instruct-nf4 is a machine learning model from bschooled. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
4-bit NF4 (bitsandbytes) quantization of Qwen/Qwen2.5-0.5B-Instruct.
All-time downloads
32
Public
Parameters
494M
457 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors457 MB · 98%
Parameter types
How the weights are stored.
U8358M · 72%
Model card
From the Hugging Face model README
qwen2.5-0.5b-instruct-nf4
4-bit NF4 (bitsandbytes) quantization of
Qwen/Qwen2.5-0.5B-Instruct.
| Upstream repository | Qwen/Qwen2.5-0.5B-Instruct |
| Upstream revision | 7ae557604adf67be50417f59c2c2f167def9a775 |
| Upstream license | apache-2.0 |
| Quantization | bitsandbytes NF4, double quantization, bf16 compute |
| Original size | 0.93 GiB |
| Quantized size | 0.44 GiB |
Text LLM used for the chat/completions path.
Changes from upstream
The weight tensors have been replaced with their NF4-quantized equivalents and a
quantization_config block was added to config.json. Tokenizer, processor,
and any remote-code files are copied unmodified from the upstream revision above.
No architecture, vocabulary, or generation defaults were altered.
Requirements
Loading requires bitsandbytes and a supported accelerator. NF4 kernels are
available for NVIDIA sm_75+ (Turing and newer) and for AMD ROCm on RDNA3 /
RDNA3.5 / CDNA.
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"bschooled/caeleste-speech",
subfolder="qwen2.5-0.5b-instruct-nf4",
device_map={"": "cuda:0"},
)
License
This derivative is distributed under the upstream license, apache-2.0.
All upstream copyright, patent, trademark and attribution notices are retained.
Refer to the upstream repository for the authoritative license text.
- Type
- qwen2
- Created
- Aug 18, 2026
- Updated
- Aug 18, 2026