Qwen3-1.7B-ExecuTorch

Qwen3-1.7B for ExecuTorch

The int4 codes were solved on 2026-09-19. The earlier files here rounded their int4 weights to the nearest grid point. This export keeps the same format, size and kernels but takes its codes from a GPTQ solve on the delegate's own grid (symmetric, groups of 32, scales exactly representable in bf16, each layer's Hessian taken after the per-token int8 rounding the delegate applies). On a Galaxy S25 Ultra through Firebase Test Lab it passes 27 of 30 BFCL tool-calling prompts where the rounded export passed 24; GSM8K and IFEval were 22 and 14 of 30 on the same run. Unlike LFM2.5, Qwen3 kept most of its tool calling under rounding, so this is an improvement rather than a repair. Method and numbers: a compiled LFM2.5 that calls tools.

The same weights exported 2 times, differing only in the context window compiled into the static KV cache. Exported with ExecuTorch 1.4.0's own recipe for the family (examples/models/qwen3, extension.llm.export.export_llm): int8 dynamic activations, int4 weights in groups of 32 (the layout Arm KleidiAI kernels take in the XNNPACK runtime), int8 embedding table, prefill chunk 2048 tokens. Runs in any app on the ExecuTorch 1.4.0 Android or iOS LLM runtime; built for OpenWeights.

Pick by memory, not speed: the runtime attends over filled positions only, so a larger window costs memory at load and nothing per token (measured: every window within about 1% of the smallest on prefill and decode in a paired probe). The 32k file needs about 7 GB of KV cache on top of the weights; on 12 GB phones (Galaxy S24+, S25 Ultra, Poco X8 Pro) the process was killed after a few prompts, by Samsung's Heimdall guard and by MIUI's ActivityManager alike, both at a 6 GB threshold; only a 16 GB Pixel 10 Pro XL ran it. Use 2k on a 12 GB phone. Measured on a Dimensity 9400 (Poco X8 Pro), awake, median of six turns: see docs/research/window-matrix.md. The full window matrix on four chips is in the OpenWeights repository under docs/research/window-matrix.md.

tokenizer.json is the base model's. EOS 151645. Chat template: the base model's ChatML-style template (<|im_start|>role ... <|im_end|>).

Weights are Alibaba Cloud's Qwen3-1.7B, Apache-2.0; this repository only changes their packaging.