llama-3.2-3b-instruct-mllm-qnn-htp-conversion-artifacts-ctx2048

Llama-3.2-3B-Instruct MLLM QNN HTP artifacts (context 2048)

Archived conversion artifacts prepared for an Android MLLM serialized replay. They are hardware- and runtime-specific and are not general Hugging Face Transformers checkpoints.

SoC mapping

• SM8650 = Qualcomm Snapdragon 8 Gen 3, Qualcomm Hexagon HTP v75 (the Redmi K70 Pro platform).
• SM8750 = Qualcomm Snapdragon 8 Elite, Qualcomm Hexagon HTP v79 (the Realme RMX5090 platform).

Why this is a separate MLLM repository

This repository is the conversion backup for Meta Llama 3.2 3B Instruct. It contains intermediate .mllm, safetensors, calibration, and metadata files used to produce the final QNN contexts. The companion llama-3.2-3b-instruct-mllm-qnn-htp repository is the deployable package with the final per-SoC context binaries and benchmark results. The two repositories therefore represent different stages of the same model, not two separate 3B models.

Export parameters

• Source model: local mirror of meta-llama/Llama-3.2-3B-Instruct
• MLLM source base commit: 50ad5a9b6fbea742e38b5b31776c187e50319c8e plus the local Llama QNN AOT/context-2048 changes used by this experiment
• QNN SDK build: v2.40.0.251030114326_189385
• PTQ calibration: 128 samples, maximum sequence length 2048
• Calibration validation: all 56 Llama RoPE ConcatObservers had matching branch scale and zero point (ConcatObserver validation PASSED)
• RoPE sin/cos table shape: [1, 2048, 128], stored as uint16
• Weight/lm-head fallback: LPBQ symmetric W4A16, block size 32
• KV cache: symmetric per-tensor W8A8
• Graphs in each context: SHA prefill sequence 32 and decode sequence 1
• Total compiled context: 2048
• QNN performance/session: HtpBurst, unsigned PD, 8 MiB VTCM capability
• Targets:
  • SM8650 / HTP V75 (Redmi experiment device)
  • SM8750 / HTP V79 (Realme experiment device)
• Runtime decoding: greedy; MLLM_QNN_IO_MEM_ESTIMATION=1

The PTQ path disabled fake quantization during calibration, recomputed scale and zero point, validated concatenation observers, ran an 8-token sanity inference, converted the model, saved safetensors, and converted it to MLLM v2. The AOT compiler then sliced MHA projections into per-head SHA weights and compiled both static graphs into one context per SoC.

Files and checksums

Both AOT logs completed with two ALLOCATE SUCCESSFUL markers, two QnnGraph_finalize done. status 0x0 markers, and the final SHA compilation completed successfully marker.

Limitations

These binaries require the matching Qualcomm QNN 2.40 runtime, the MLLM runtime built from the stated source plus local changes, and matching V75/V79 custom op packages. They are intended for internal reproducibility and have not been evaluated as general-purpose model releases.