qwen3.5-8b-distilled-MLX

Qwen3.5-8B Distilled - MLX Format

Fine-tuned Qwen3.5-8B for software engineering & coding tasks. MLX-optimized for Apple Silicon (M1/M2/M3/M4).

🎯 Apple Silicon Native

• Optimized for Mac M1, M2, M3, M4
• Direct memory access (no GPU bottleneck)
• Native performance, minimal overhead

🚀 Quick Start

MLX (Mac)

from mlx_lm import load, generate

model, tokenizer = load("nachikethreddyy/qwen3.5-8b-distilled-MLX")

prompt = "Write a Python function to"
response = generate(
    model, tokenizer, 
    prompt=prompt, 
    max_tokens=200, 
    verbose=True
)
print(response)

Installation

pip install mlx-lm

📊 Model Info

• Size: ~31 GB (4-bit effective)
• Format: MLX safetensors
• Base: Qwen/Qwen3-8B
• Fine-tuning: LoRA (r=16, alpha=32)
• Data: 256 coding examples

📊 Training Details

• Framework: MLX (Apple Silicon optimized)
• Training Time: ~5 hours on Mac M4 Pro
• Max Sequence: 1536 tokens
• Iterations: 1600

💡 Performance

• Memory: ~8-16GB (shared with system)
• Speed: Fast token generation on Apple Silicon
• Quality: Maintained through 4-bit quantization

📄 License

Apache 2.0 (inherited from Qwen/Qwen3-8B)

---

For GGUF/Local inference: See qwen3.5-8b-distilled-GGUF