gemma4-e2b-q6a-qcs6490-litertlm-npu

Gemma4 E2B Q6A/QCS6490 ctx1024 LiteRT-LM NPU Artifact

This repository hosts a .litertlm model container prepared for an unofficial Radxa Dragon Q6A / Qualcomm QCS6490 LiteRT-LM NPU route.

This is not an official Google, Radxa, or Qualcomm release.

Artifact

• File: gemma4_e2b_q6a_qcs6490_ctx1024_npu.litertlm
• SHA256: 0d72585954b3855df20735b666926b5617c0e4a43528be6c00abe06fa0812347
• Target board: Radxa Dragon Q6A
• Target SoC family: Qualcomm QCS6490 / QCM6490
• Tested RAM: 12 GB
• Tested OS: Ubuntu 24.04.4 LTS
• Tested QAIRT/QNN runtime: 2.42.0.251225135753_193295
• Practical route: ctx1024 Direct NPU
• Runtime requirement: patched LiteRT-LM runtime from the companion GitHub repository
• Tested LiteRT-LM source commit for the companion runtime patch: 497e7e28bd89c2b4e0d88e75035b045a21bc33fa

What This Is

This artifact is a Q6A/QCS6490-oriented LiteRT-LM .litertlm package derived from Gemma 4 E2B / the LiteRT-LM Gemma4 artifact family. It is intended to be used with the patched runtime and wrapper documented in the companion GitHub repository:

https://github.com/PrismPhi/radxa-dragon-q6a-qcs6490-gemma4-litertlm-npu

The GitHub repository contains runtime patches, scripts, reproduction notes, and license/distribution notes. It does not contain this .litertlm file. Use the runtime build instructions there rather than a stock upstream litert_lm_main.

What This Is Not

This repository does not include:

• Qualcomm QAIRT/QNN runtime libraries
• libQnn*.so
• libLiteRtDispatch_Qualcomm.so
• libGemmaModelConstraintProvider.so
• LiteRT-LM runtime binaries
• Gemma safetensors or original training weights

Users must provide Qualcomm runtime libraries from their own properly licensed Q6A/QAIRT/Qualcomm AI Runtime environment.

License And Attribution

This artifact follows the Apache-2.0 licensing lineage of the published Gemma 4 E2B model and LiteRT-LM Gemma4 artifact family.

Relevant upstream references:

• Base model: https://huggingface.co/google/gemma-4-e2b-it
• LiteRT-LM Gemma4 artifact family: https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm
• LiteRT-LM source: https://github.com/google-ai-edge/LiteRT-LM

This repository includes LICENSE, NOTICE, and SHA256SUMS. If you use or redistribute this artifact, also follow the upstream Gemma and LiteRT-LM terms and any Qualcomm/Radxa licensing terms for runtime libraries you use locally.

Runtime Layout

Recommended Q6A layout:

~/q6a-gemma4-npu/
  packages/
    gemma4_e2b_q6a_qcs6490_ctx1024_npu.litertlm
  runtime/
    litert_lm_main_q6a_gemma4_npu
    q6a-gemma4-npu
    libGemmaModelConstraintProvider.so
    libLiteRtDispatch_Qualcomm.so
    libQnnHtp.so
    libQnnHtpPrepare.so
    libQnnHtpV68.so
    libQnnHtpV68Skel.so
    libQnnHtpV68Stub.so
    libQnnSystem.so

Smoke Test

After installing the companion GitHub repository and preparing runtime libraries:

printf '%s' 'MSsx44Gv77yf55+t44GP562U44GI44Gm' | base64 -d > /tmp/q6a_prompt.txt

/home/radxa/bin/q6a-gemma4-npu run \
  ~/q6a-gemma4-npu/packages/gemma4_e2b_q6a_qcs6490_ctx1024_npu.litertlm \
  --backend npu \
  --temperature 0 \
  --prompt-file /tmp/q6a_prompt.txt

Expected answer:

2

Expected route marker:

Q6A_GEMMA4_NPU_ROUTE_SELECTED=ctx1024_direct_q6a_package

Performance Notes

In the original Q6A project, the ctx1024 Direct NPU route was the practical default. It was faster than the longer ctx4096 Direct route for short and medium prompts, while the ctx4096 route remained useful as a long-context fallback. Full Gemma4 decode/verify QNN-context execution was investigated but was not adopted as the fastest practical route.

See the companion GitHub repository for detailed benchmark notes.

Project Provenance

This artifact and its documentation were prepared as part of an experimental Codex-assisted engineering project. The result should be treated as an unofficial community artifact for reproducibility and further experimentation.