FastContext-1.0-4B-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta, 2026-06).

Mirror of mlboydaisuke/FastContext-1.0-4B-CoreAI — the canonical repo (CoreAI Model Zoo). Updates land there first.

FastContext-1.0-4B-SFT — Core AI (on-device, iPhone)

microsoft/FastContext-1.0-4B-SFT converted to Apple Core AI for on-device inference, served by the CoreAIChat app.

FastContext is a long-context repository-exploration agent (Qwen3-4B-Instruct backbone, supervised-fine-tuned on exploration traces: broad first-turn search, multi-turn evidence gathering, and precise file:line citation generation).

Contents

• gpu/ — AOT-compiled GPU (MPSGraph) bundle, 4-bit linear-INT4, h18p (iPhone 17 / iPhone 18 class). ~2.1 GB. Drop-in for CoreAIChat's Documents/models/fastcontext_4b_gpu.

Device class: this is an AOT h18p bundle (iPhone 17 / 18 class). On-device specialization of a 4B graph is not viable, so the bundle is precompiled for the h18p GPU — the same approach Core AI uses for the Gemma-4B bundle.

Verified on iPhone 17 Pro (iOS 27)

• Parity vs HF fp16 (teacher-forced): 23/24 argmax, perplexity 1.41 — the single flip is a near-tie quant-noise position.
• On-device GPU: ~334 tok/s prefill, ~16.5 tok/s decode (512-prompt / 1024-gen), load ~8 s cold, ~0.3 GB resident (weights memory-mapped).

Use

Install CoreAIChat, open the model picker, and select FastContext 4B.

License

MIT, inherited from the base model microsoft/FastContext-1.0-4B-SFT. See LICENSE.