hovor-qwen3-1.7b-coreml-int4

hovor-qwen3-1.7b-coreml-int4

CoreML conversion of Qwen/Qwen3-1.7B for Apple Neural Engine (ANE) inference on iOS/macOS.

Details

• Source model: Qwen/Qwen3-1.7B (Apache-2.0)
• Architecture: qwen3 (28 layers, 2048 hidden size, GQA 16/8 heads, QK-norm)
• Quantization: int4 palettization (group_size=32) via coremltools 8.3
• Context length: 512 tokens
• Package size: ~971 MB
• Deployment target: iOS 18+ / macOS 15+ (MLState stateful KV cache)
• ANE safety: 0 GPU-preferred ops — safe for iOS background (CPU+ANE only)

Files

• model.mlpackage — CoreML stateful transformer (int4 palettized weights)
• model_config.json — metadata for CoreMLLLM Swift inference engine
• hf_model/ — tokenizer files (safetensors weights excluded to keep repo small)

Usage

Use with the CoreML-LLM Swift package.

// model_config.json tokenizer_repo points to Qwen/Qwen3-1.7B
// Chat format: ChatML (<|im_start|>role\n…<|im_end|>\n)
// Thinking mode is ON by default; suppress with /no_think in user message

License

Apache-2.0 (same as source model). Conversion tooling (CoreML-LLM) is MIT.