QwenPaw-Flash-4B-coreml

Model Card for QwenPaw-Flash-4B (Core ML, Apple Neural Engine)

A compiled Core ML build of QwenPaw-Flash-4B, packaged to run entirely on device on the Apple Neural Engine (ANE). It targets macOS (Apple silicon), needs no network at inference time, and streams tokens locally with the weights quantized to 6 bits.

This repository holds the compiled model set only, not training or original weights. It is a self-contained artifact that a companion on-device runtime loads to drive generation. Nothing here calls out to a server.

Model Details

Model Description

QwenPaw-Flash-4B shares the Qwen3.5 hybrid design: of its 32 layers, 24 are Gated DeltaNet linear-attention (state-space) blocks and 8 are full softmax-attention blocks, interleaved "three linear, one attention". The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall. It is the larger sibling of the 0.8B and 2B builds, with the same vocabulary and a deeper, wider network. This build converts the upstream model to Core ML and palettizes the weights to 6 bits for on-device use.

• Developed by: AgentScope / QwenPaw authors (base model); Core ML conversion by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model, converted to Core ML (mlprogram)
• Language(s): English and the languages of the base model
• License: Apache 2.0 (inherited from the base model)
• Converted from model: agentscope-ai/QwenPaw-Flash-4B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/QwenPaw-Flash-4B-coreml
• Base model: agentscope-ai/QwenPaw-Flash-4B (Apache 2.0)

Uses

Direct Use

On-device assistants and chat, drafting, and summarization, in offline or privacy-sensitive settings where prompts must not leave the device. At 4B parameters it is more capable than the 0.8B and 2B builds but heavier: it is intended for Apple silicon Macs with adequate memory, not phones. The set is driven by an on-device runtime that runs prompt ingestion (prefill) as a batched pass on the Neural Engine and generates tokens one at a time, carrying the linear-attention state forward and reading the attention layers from a paged key/value cache.

Out-of-Scope Use

Not for tasks requiring verified factual accuracy, and not a basis for legal, medical, financial, or safety decisions. Not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

A language model predicts plausible text; it does not verify facts and has no understanding of truth. Output can be confident and still be wrong, incomplete, or biased, and 6-bit quantization can widen small errors relative to the base model. The base model's biases and limitations carry over; see the base model card.

Recommendations

Treat every response critically and check important output against a trusted source. This model defaults to emitting a "thinking" prefix; preserve that per-model default when integrating a chat template.

How to Get Started with the Model

The repository contains a set of compiled .mlmodelc programs plus the tokenizer. It is designed to be loaded by an on-device runtime that implements paged key/value attention on the host between the Neural Engine trunk programs; it is not a single .mlpackage you call with one prediction. The included chat_template.jinja documents the chat format (roles, the "thinking" prefix, tool-call rendering).

Technical Specifications

Model Architecture and Objective

Files

• mf0of9.mlmodelc ... mf8of9.mlmodelc - the transformer trunk, cut into 9 programs around the attention layers; each is a multifunction bundle whose "prefill" (batched) and "decode" (single-token) graphs share one quantized weight blob, so the weights are resident once.
• head.mlmodelc - the tied vocabulary matrix, shared by the output projection and the token-embedding lookups (one quantized copy, three functions).
• tile.mlmodelc, tile_prefill.mlmodelc, tile_carry.mlmodelc - weightless flash-attention kernels for the paged key/value cache.
• tokenizer.json, tokenizer_config.json, chat_template.jinja - the byte-level BPE tokenizer and chat template.

Compute Infrastructure

• Hardware: Apple silicon Mac with a Neural Engine.
• Software: macOS 15+. The first launch compiles the programs for the Neural Engine once; the OS caches that build, so later launches are fast.

Citation

Please cite the upstream QwenPaw-Flash-4B model. See the base model card at agentscope-ai/QwenPaw-Flash-4B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.