Qwen3-Embedding-0.6B-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) -->

This model has no row on DeviceMark, the on-device LLM leaderboard.

<!-- gen-cards:devicemark end -->

Qwen3-Embedding-0.6B — Core AI export

Qwen/Qwen3-Embedding-0.6B as a single static Core AI graph for macOS 27 / iOS 27: the full sentence-transformers pipeline (Qwen3-0.6B backbone → last-token pooling → L2 normalize) runs in-graph, so one call returns a normalized, MRL-truncatable 1024-d embedding. Multilingual (incl. Japanese), instruction-aware on-device semantic search / RAG.

This is an encoder — one forward over the (right-padded) input → one pooled vector. No autoregressive loop, no KV cache, no LM head. It runs as a plain .aimodel via AIModel.run (like the vision encoders), not the pipelined generate engine.

Graph contract

The grid (512) is an export-time choice — a smaller grid is proportionally faster for short queries. Last-token pooling under the causal mask is right-pad safe (real tokens never attend to trailing pads), so the host just right-pads to the grid.

Host recipe (everything else is in-graph)

• Query → prepend the instruction prefix: Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: Document → no prefix.
• Tokenize, right-pad to 512 (truncate longer text). Run → 1024-d unit vector.
• Similarity = cosine = dot product (vectors are unit-norm).
• Matryoshka (MRL): to shrink, take the first D dims (32 ≤ D ≤ 1024) and re-L2-normalize on the host. Rankings are preserved down to 256; verified to 128.

# Core AI runtime (Python), GPU delegate
import coreai.runtime as rt, numpy as np
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("tokenizer")
m = await rt.AIModel.load("qwen3-embedding-0.6b_float16_s512_static.aimodel",
        rt.SpecializationOptions.from_preferred_compute_unit_kind(rt.ComputeUnitKind.gpu()))
fn = m.load_function("main")

def embed(text, is_query):
    prefix = ("Instruct: Given a web search query, retrieve relevant passages that "
              "answer the query\nQuery:") if is_query else ""
    enc = tok(prefix + text, padding="max_length", truncation=True, max_length=512,
              return_tensors="np", padding_side="right")
    res = await fn({"input_ids": rt.NDArray(enc["input_ids"].astype(np.int32)),
                    "attention_mask": rt.NDArray(enc["attention_mask"].astype(np.int32))})
    return res["embedding"].numpy()[0]   # [1024], unit-norm

Swift — CoreAIKit

Downloads this repo on first use and applies the prompts in-process:

import CoreAIKitEmbeddings

let embedder = try await TextEmbedder(model: .qwen3Embedding0_6B, prompts: .qwen3Embedding)
let query = try await embedder.embed(query: "What is the capital of Japan?")
let doc   = try await embedder.embed(document: "Tokyo is the capital and largest city of Japan.")
let score = TextEmbedder.cosineSimilarity(query, doc)   // unit vectors → dot product = cosine

Bundle layout

qwen3-embedding-0.6b_float16_s512_static.aimodel   (~1.1 GB, fp16)
tokenizer/                                          (HF tokenizer files)
reference.json                                      (torch reference embeddings + cosines)

Parity

Precision fp16. Verified against the official sentence-transformers pipeline (fp32): per-text embedding cosine 1.000000, retrieval order identical, MRL rankings preserved at 512 / 256 / 128. On the Core AI GPU delegate the .aimodel reproduces the torch reference at cosine 0.999998 end-to-end (host tokenize → run). Measured ~25 ms (256-grid) / ~45 ms (512-grid) per embedding on an M4 Max GPU.

License

Apache-2.0 (upstream model and code are Apache-2.0). Conversion script: conversion/export_qwen3_embedding.py in the coreai-model-zoo.

<!-- funnel:v1 -->

---

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->