Qwen3.5-0.8B-coreml

Model Card for Qwen3.5-0.8B (Core ML, Apple Neural Engine)

A compiled Core ML build of Qwen3.5-0.8B, packaged to run entirely on device on the Apple Neural Engine (ANE). It targets macOS and iOS, needs no network at inference time, and streams tokens locally with the weights quantized to 6 bits.

This repository holds the compiled model set only, not training or original weights. It is a self-contained artifact that a companion on-device runtime loads to drive generation. Nothing here calls out to a server.

Model Details

Model Description

Qwen3.5-0.8B is a hybrid decoder: of its 24 layers, 18 are Gated DeltaNet linear-attention (state-space) blocks and 6 are full softmax-attention blocks, interleaved in a repeating "three linear, one attention" pattern. The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as the context grows, while the periodic attention blocks preserve exact long-range recall. This build converts the upstream model to Core ML and palettizes the weights to 6 bits for on-device use.

• Developed by: Qwen team (base model); Core ML conversion by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model, converted to Core ML (mlprogram)
• Language(s): English and the languages of the base model
• License: Apache 2.0 (inherited from the base model)
• Converted from model: Qwen/Qwen3.5-0.8B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/Qwen3.5-0.8B-coreml
• Base model: Qwen/Qwen3.5-0.8B (Apache 2.0)

Uses

Direct Use

On-device assistants and chat, drafting, and summarization, in offline or privacy-sensitive settings where prompts must not leave the device. It is a small model: best for short, focused tasks rather than long, open-ended reasoning. The set is driven by an on-device runtime that runs prompt ingestion (prefill) as a batched pass on the Neural Engine and generates tokens one at a time, carrying the linear-attention state forward and reading the attention layers from a paged key/value cache.

Out-of-Scope Use

Not for tasks requiring verified factual accuracy, and not a basis for legal, medical, financial, or safety decisions. Not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

A language model predicts plausible text; it does not verify facts and has no understanding of truth. Output can be confident and still be wrong, incomplete, or biased, and 6-bit quantization can widen small errors relative to the base model. The base model's biases and limitations carry over; see the base model card.

Recommendations

Treat every response critically and check important output against a trusted source. Preserve the model's per-model default behavior for the "thinking" prefix when integrating a chat template.

How to Get Started with the Model

The repository contains a set of compiled .mlmodelc programs plus the tokenizer. It is designed to be loaded by an on-device runtime that implements paged key/value attention on the host between the Neural Engine trunk programs; it is not a single .mlpackage you call with one prediction. The included chat_template.jinja documents the chat format (roles, the "thinking" prefix, tool-call rendering).

Technical Specifications

Model Architecture and Objective

Files

• mf0of7.mlmodelc ... mf6of7.mlmodelc - the transformer trunk, cut into 7 programs around the attention layers; each is a multifunction bundle whose "prefill" (batched) and "decode" (single-token) graphs share one quantized weight blob, so the weights are resident once.
• head.mlmodelc - the tied vocabulary matrix, shared by the output projection and the token-embedding lookups (one quantized copy, three functions).
• tile.mlmodelc, tile_prefill.mlmodelc, tile_carry.mlmodelc - weightless flash-attention kernels for the paged key/value cache.
• tokenizer.json, tokenizer_config.json, chat_template.jinja - the byte-level BPE tokenizer and chat template.

Compute Infrastructure

• Hardware: Apple silicon with a Neural Engine (M-series Mac, or an iPhone/iPad).
• Software: macOS 15+ / iOS 18+. The first launch compiles the programs for the Neural Engine once; the OS caches that build, so later launches are fast.

Citation

Please cite the upstream Qwen3.5 model. See the base model card at Qwen/Qwen3.5-0.8B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.