Downloads · 30 days
13
7% of all-time downloads
EdgeReason-Qwen2-0.5B-GGUF
ajiteshdeb/EdgeReason-Qwen2-0.5B-GGUF
EdgeReason-Qwen2-0.5B-GGUF is a text generation model from ajiteshdeb. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a 4-bit quantized (Q4KM) version of a 0.5B parameter Student model trained via Knowledge Distillation from a 1.5B parameter Teacher model.
All-time downloads
190
Public
Repo size
398 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf398 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Jul 18, 2026
- Updated
- Jul 18, 2026
- SHA
- a636ade2
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
This is a 4-bit quantized (Q4_K_M) version of a 0.5B parameter Student model trained via Knowledge Distillation from a 1.5B parameter Teacher model.
Model Details
- Teacher Model: Qwen2-1.5B-Instruct
- Student Model: Qwen2-0.5B-Instruct
- Training Dataset: GSM8K (Math Reasoning)
- Quantization: Q4_K_M via llama.cpp
Distillation Methodology
The student was trained from scratch using a custom PyTorch loop with a combined loss function:
- Cross-Entropy Loss: Standard ground-truth labeling.
- KL Divergence Loss: Soft label transfer at Temperature 2.0.
- Attention Transfer Loss: Mean Squared Error matching internal attention maps between Teacher and Student.
Performance (Ablation Study)
Usage
This model is designed for CPU inference using llama.cpp or LM Studio.
./llama-server -m student_model_q4_km.gguf -c 2048 -ngl 99 --port 8080
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Jul 18, 2026
- Updated
- Jul 18, 2026