qwen3-embedding-0.6b-q4f16_1-MLC

Qwen3-Embedding-0.6B-q4f16_1-MLC

MLC-compiled Qwen3-Embedding-0.6B (Apache-2.0) in web-llm 0.2.84's format for WebGPU in-browser embedding. Built locally with mlc-llm (tvmjs 0.24.0-dev3 / mlc-ai 0.26.dev246 generation) + emsdk 3.1.56.

• Quantization: q4f16_1 · context: 2048 · prefill chunk: 2048 · max batch: 2
• Embedding dim: 1024 (last-token pooling, L2-normalized — done in the compiled model)
• WebGPU: works on NVIDIA; the fp16 embedding table (~310 MB) sits far under tvmjs's 1 GiB buffer cap, so no requestDevice shim is needed (unlike the 8B build)
• VRAM: ~1.5 GB (compile estimate ~1248 MB) — runs on any WebGPU-capable GPU
• Size: 9 shards, ~341 MB download (~335 MB weights) — the lightest of the Qwen3-Embedding family builds
• Why q4f16_1? same family-wide toolchain constraint: the fp8 8-bit path cannot build for WebGPU (TVM's wasm codegen only emits f16/f32), so q4f16_1 is the effective VRAM-friendly build.

Build note: unlike the 8B checkpoint (which declares tie_word_embeddings=false but ships no lm_head.weight), the 0.6B is genuinely tied — the tie patch is a no-op here.

Files

Use in web-llm (0.2.84)

const record = {
  model: 'https://huggingface.co/skkut/qwen3-embedding-0.6b-q4f16_1-MLC/resolve/main/',
  model_id: 'Qwen3-Embedding-0.6B-q4f16_1-MLC',
  model_lib: 'https://huggingface.co/skkut/qwen3-embedding-0.6b-q4f16_1-MLC/resolve/main/qwen3-embedding-0.6b-q4f16_1-ctx2048_batch2-webgpu.wasm',
  model_type: 1, // EmbeddingModelType
  vram_required_MB: 1500,
};

Query-side texts should use the Qwen3 instruction prefix (Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: …). On the 0.6B the prefix's absolute cosine gain is smaller than on the 4B/8B (typical family guidance: 1–5%) — smoke-verified retrieval ranking stays correct across a mixed EN/ZH probe set either way.