litert-lm-unity-quantized

Community Release — Project-Quantized LiteRT Models (2026-07-23)

Quantized ASR/LLM artifacts produced by the LiteRT-LM-Unity project, prepared for upstream contribution (e.g., litert-community on Hugging Face). All files were validated before inclusion; validation data below.

Toolchain / provenance

• Quantizer: ai-edge-quantizer 0.8.0 (post-training, dynamic-range).
• Recipes:
  • i8 = recipe.dynamic_wi8_afp32() (int8 weights channelwise, fp32 activations).
  • i4 = recipe.dynamic_wi4b64_afp32() (int4 weights, blockwise-64, fp16 scales) with i8 overrides on sensitive scopes (encoder / embedding-logits table / selected projections). Pure full-scope int4 was tested and REJECTED for several models (Korean transcription errors); the mixed recipes below are what passed.
  • int4 channelwise (wi4c) and blockwise-32 (wi4b32) are not used in this collection: in our tests wi4c degraded accuracy sharply and wi4b32 produced an immediate EOS. Do not regenerate with them.
• Whisper medium / large-v3 / large-v3-turbo tflite graphs were exported by this project from the openai checkpoints (30 s fixed window, encode/decode split signatures matching the litert-community whisper-tiny/base graph interface; large-v3 family = 128 mel bins / 51866 vocab). f32 source exports (3–6 GB) are retained separately and available on request.
• Qwen2.5 .litertlm files were produced by unpack → quantize → repack: litert-lm-peek dump → dynamic_wi4b64_afp32 on the f32 tflite section → rebuild with litert-lm-builder 0.14.0 (tokenizer + LlmMetadata mirrored from the official q8 release). Source: official litert-community f32 releases.
• Runtime compatibility: .litertlm format 1.5.0; validated on LiteRT-LM v0.14.0 (Windows x86_64 CPU/GPU and Android arm64, Snapdragon 865-class device).

Files and validation

Accuracy-claim scope: "exact" below means the specific gate clips used during quantization validation (sentence-length Korean/English test audio), not general Korean capability. Per-tier CER across the full 10-clip matrix — including short voice commands, where small tiers degrade sharply — is published in the project repo's docs/benchmarks/asr-model-matrix.md.

Validation clips: Korean "2025년 3월 5일 전술평가 결과 보고" sentence + English equivalent + short Korean voice commands ("볼륨 업" etc.). CER computed against reference transcripts (punctuation-normalized). Full matrix: docs/benchmarks/asr-model-matrix.md in the LiteRT-LM-Unity repo.

Related: ACFT short-window Whisper models

Fixed 5 s/10 s/30 s short-window TFLite graphs (encode-signature-sized mel window; method: futo-org/whisper-acft) live in two dedicated repos:

• Pure ACFT (TFLite conversions of the futo-org checkpoints, no retraining; tiny / base / small × multilingual / .en, dynamic-range int8): litert-community/whisper-acft.
• Korean ACFT (ACFT-KO, Korean-trained for on-device Korean voice commands; tiny / base / medium / large-v3-turbo, dynamic-range int8): leuconoe/whisper-acft-ko. Training data zeroth-korean (OpenSLR SLR40, CC-BY-4.0) + google/fleurs en_us (CC-BY-4.0).

Known caveats (disclose in model cards)

• Whisper 30 s graphs; decoder is fixed-length re-run (no KV cache) matching the litert-community tiny/base interface. large-v3 family needs a 128-mel frontend and vocab 51866 (<|transcribe|>=50360, <|notimestamps|>=50364).
• tiny/base i8 showed sub-1.2 s clip sensitivity on one Snapdragon 865 device (mel/STFT numerics, ~0.1 % energy delta vs desktop); turbo/large-v3 unaffected.
• Qwen2.5 i4 litertlm metadata carries lowercase model_type (tf_lite_prefill_decode) vs uppercase in official files — loads fine on v0.14.0, noted for completeness.
• Licenses: whisper weights MIT (OpenAI); Qwen2.5 Apache-2.0. Quantized derivatives inherit the base licenses.