qwen3-1.7b-executorch-qnn-htp

Qwen3-1.7B — ExecuTorch QNN/HTP (SM8650 + SM8750)

Two ExecuTorch .pte artifacts of Qwen3-1.7B compiled for the Qualcomm Hexagon NPU (HTP), plus the complete 160-request latency measurement taken with them on two physical phones.

Both artifacts are fully measured: 160/160 requests completed on each device, with per-token device timestamps.

SoC mapping

• SM8650 = Qualcomm Snapdragon 8 Gen 3, Qualcomm Hexagon HTP v75 (the Redmi K70 Pro platform).
• SM8750 = Qualcomm Snapdragon 8 Elite, Qualcomm Hexagon HTP v79 (the Realme RMX5090 platform).

Artifacts

Both files carry the FlatBuffer identifier ET12 at bytes 4–7 and contain 75 execution plans. Verified graph shapes (read back from the serialized program, not from the build log):

prefill_forward : input_tokens [1, 128]   attention_mask [1, 1, 128, 2048]
kv_forward      : input_tokens [1,   1]   attention_mask [1, 1,   1, 2048]

The filename hybrid_llama_qnn.pte is what upstream llama.py emits for --model_mode hybrid; the llama in the name is the shared decoder harness, not the model. This is Qwen3.

Export parameters

Quantization

Toolchain

QNN SDK 2.40.0 (build_id 251030114326_189385), host torch 2.9.1+cu128, --compile_only (compilation is host-side; no AI Hub job involved).

Measured latency

Serial FIFO replay, one resident runner per device, 160 requests executed contiguously with no restart. Greedy decoding, context 2048. TTFT is model-side; TPOT is the mean inter-token interval derived from adjacent device commit timestamps.

Per-dataset TTFT P50 / TPOT P50 (ms):

SLO attainment (against the per-request TTFT/TPOT targets carried in the workload): Redmi 0.3812 / 0.2301, Realme 0.3875 / 0.2398.

Workload

160 requests, 40 each from gsm8k / humaneval / math500 / mt_bench. Prompts are pre-tokenized and replayed as input_ids, so no chat template is applied twice. max_new_tokens is per-request from the workload (628–1849).

Result files

Known metadata defect in the Realme raw result. The deployed replay runner wrote soc_model=SM8650 unconditionally on every row. The actual target was SM8750/v79 with qwen3_1_7b_hybrid_sm8750.pte. result_full.jsonl is the untouched device output and is kept as-is; result_full.sm8750.jsonl changes only the soc_model field. An object-by-object comparison after removing that key confirmed every other field is identical. Reports were generated from both.

Each result row was validated for: exact workload request order, status=ok, completion_tokens == output_tokens == len(token_trace), contiguous token_index, and strictly increasing device_commit_ns.

Reproduction

python executorch/examples/qualcomm/oss_scripts/llama/llama.py \
  --artifact <out> --build_folder <android-build> \
  --soc_model SM8650 --backend htp \
  --decoder_model qwen3-1_7b \
  --checkpoint qwen3-1.7b-meta.pth \
  --params executorch/examples/models/qwen3/config/1_7b_config.json \
  --tokenizer_model <Qwen3-1.7B>/tokenizer.json \
  --model_mode hybrid --prefill_ar_len 128 \
  --max_seq_len 2048 --max_context_len 2048 \
  --temperature 0 --compile_only

On device, via qnn_llama_replay_runner:

--decoder_model_version qwen3 --eval_mode 1 \
--max_context_len 2048 --temperature 0

ADSP_LIBRARY_PATH must include the runner directory alongside the platform DSP paths, or QNN device creation fails with error 14001.

Scope

These artifacts were produced for a serial-arrival, single-request latency comparison across on-device inference frameworks. Batch size is 1 and there is no continuous batching or speculative decoding. Accuracy was not evaluated — only latency and token-trace integrity.