qwen2.5-0.5b-pruned-distilled-game

Qwen2.5-0.5B — Pruned & Distilled for Real-Time Character Dialogue

A compressed derivative of Qwen/Qwen2.5-0.5B-Instruct (Apache 2.0), built as part of a project exploring real-time inference optimization for interactive virtual characters.

What was done

1. Layer importance was measured empirically, not guessed — each of the 24 transformer layers was individually bypassed and the resulting change in output was measured via cosine similarity against the unmodified model. This produced a ranked list of which layers were safest to remove.
2. 5 layers were removed (the top empirically-ranked candidates), reducing the model to 19 layers. This alone broke the model — it became confused about its own role and failed to follow basic instructions, despite being built from the "safest" layers by the ranking.
3. Knowledge distillation recovered it. The pruned model was fine-tuned (KL-divergence loss, temperature-scaled soft labels) against the original 24-layer model as teacher, using a small set of teacher-generated examples. This is what actually produced this checkpoint — coherent output, recovered from what was previously broken.

Benchmarks (CPU, fp32)

~35% faster than the original model, with quality restored via distillation rather than avoided by pruning less aggressively.

Intended use

Built for a real-time virtual character inference pipeline (text dialogue → TTS → game engine integration). Small-scale, CPU-benchmarked; not evaluated for general-purpose task performance beyond conversational coherence.

License

Apache 2.0, inherited from the base model. This is a derivative work of Qwen2.5-0.5B-Instruct — see the original model's license for full terms.