qwen3_0_6b_genie_sa8775_ctx256

Qwen3 0.6B Genie SA8775 HTP Bundle

This directory is an offline Genie/QNN bundle for running Qwen3-0.6B on SA8775P ADP HTP.

Build Summary

• Source model: Qwen/Qwen3-0.6B
• Runtime target: Qualcomm Genie / QNN HTP
• Device target: SA8775P ADP, Android 14, SoC model 52, HTP v73
• Quantization: W4A16
• Context length: 256
• Prompt processor sequence length: 128
• Token generator sequence length: 1
• HTP core: core_id = 1 in htp_backend_ext_config.json

Files

• qwen3_0_6b_ctx256_part_1_of_2.bin: linked QNN context binary, part 1
• qwen3_0_6b_ctx256_part_2_of_2.bin: linked QNN context binary, part 2
• genie_config.json: config for genie-t2t-run
• text-generator.json: config for Genie SDK text generator pipeline
• htp_backend_ext_config.json: HTP backend extension config
• tokenizer.json, tokenizer_config.json, config.json: tokenizer/model config copied from checkpoint
• sample_prompt.txt: sample prompt generated by the QAI Hub Models helper
• metadata.json: bundle metadata

Push To Device

Adjust SERIAL if multiple devices are connected. The commands below assume you run them from /Users/wangzhao/Documents/Qnnllm/files.

cd /Users/wangzhao/Documents/Qnnllm/files

SERIAL=844e658e
DEVICE_DIR=/data/local/tmp/qwen3_0_6b_genie
MODEL_DIR=qwen3_0_6b_genie_sa8775_ctx256_download
RUNTIME_DIR=qairt246_android_v73_runtime

adb -s "$SERIAL" shell "rm -rf $DEVICE_DIR && mkdir -p $DEVICE_DIR/bin $DEVICE_DIR/lib $DEVICE_DIR/dsp"

# Push the Genie model bundle.
adb -s "$SERIAL" push "$MODEL_DIR/." "$DEVICE_DIR/"

# Push executable runtime tools.
adb -s "$SERIAL" push "$RUNTIME_DIR/bin/genie-t2t-run" "$DEVICE_DIR/bin/"

# Push Android ARM-side Genie/QNN shared libraries.
adb -s "$SERIAL" push "$RUNTIME_DIR/lib/." "$DEVICE_DIR/lib/"

# Push DSP/HTP-side skeleton/runtime libraries.
adb -s "$SERIAL" push "$RUNTIME_DIR/dsp/." "$DEVICE_DIR/dsp/"

adb -s "$SERIAL" shell "chmod +x $DEVICE_DIR/bin/genie-t2t-run"

Directory meaning:

• $DEVICE_DIR/bin: executable tools, for example genie-t2t-run
• $DEVICE_DIR/lib: Android ARM-side libraries, for example libGenie.so, libQnnHtp.so, libQnnSystem.so
• $DEVICE_DIR/dsp: DSP/HTP-side libraries, for example libQnnHtpV73Skel.so, libQnnHtpV73.so

The runtime version must be compatible with the context binary and device firmware.

Run On Device

Create a prompt file:

adb -s "$SERIAL" shell "cat > $DEVICE_DIR/chat.txt <<'EOF'
<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Give me a short introduction to large language model.<|im_end|>
<|im_start|>assistant
EOF"

Run with HTP. Use semicolon separators in DSP library path values on this device family.

adb -s "$SERIAL" logcat -c
adb -s "$SERIAL" shell "cd $DEVICE_DIR && \
  export LD_LIBRARY_PATH=\$PWD/lib:\$PWD/bin:\$PWD && \
  export ADSP_LIBRARY_PATH='/vendor/lib/rfsa/adsp;$PWD/dsp;$PWD/lib' && \
  export CDSP_LIBRARY_PATH='/vendor/lib/rfsa/adsp;$PWD/dsp;$PWD/lib' && \
  export CDSP1_LIBRARY_PATH='/vendor/lib/rfsa/adsp;$PWD/dsp;$PWD/lib' && \
  timeout 180 ./bin/genie-t2t-run -c genie_config.json --prompt_file chat.txt 2>&1"

If genie-t2t-run is installed elsewhere, replace ./bin/genie-t2t-run with the actual path.

Notes

• This bundle has been compiled and linked successfully on QAI Hub. Local on-device execution still requires a matching Genie runtime binary and QNN/HTP libraries on the car unit.
• htp_backend_ext_config.json is set to core_id: 1 for SA8775 testing, because HTP0/CDSP0 has shown issues on this platform. Change it to 0 only when you explicitly want to test HTP0.
• The generated context length is small (256) to keep the first bring-up practical. Rebuild with a larger context length after the runtime path is confirmed.