neuron-kd-qwen2.5-coder-0.5b-mlx-4bit

Neuron KD — Qwen2.5-Coder-0.5B (MLX, 4-bit)

The on-device code reviewer behind the Neuron iOS app's Code Lab.

Qwen2.5-Coder-0.5B-Instruct, fine-tuned via hybrid knowledge distillation (sequence-level CE + KL on teacher logits) from Qwen2.5-Coder-32B-Instruct on Python problems from MBPP, then merged (LoRA folded into the base weights) and converted to MLX format at 4-bit (group size 64, affine).

• ~280 MB on disk · runs fully offline on iPhone via MLX Swift
• Trained to write Python solutions from a problem + one example test
• ChatML prompt format (chat_template.jinja included)

Use with mlx-lm

pip install mlx-lm
mlx_lm.generate --model Kishan25/neuron-kd-qwen2.5-coder-0.5b-mlx-4bit \
  --prompt "Problem: Write a python function to count true booleans in a list." \
  --max-tokens 200

Use with MLX Swift

let container = try await LLMModelFactory.shared.loadContainer(
    configuration: ModelConfiguration(id: "Kishan25/neuron-kd-qwen2.5-coder-0.5b-mlx-4bit")
)