Bonsai-27B-litert-lm

Bonsai-27B LiteRT-LM — source repair and experimental conversion

23 September 2026: source reconstruction is repaired and tested; a repaired full-model Android artifact is not yet available. The existing Bonsai-slice-4L-* files contain only four layers, not the complete 64-layer Bonsai-27B model. The word fixed in their historical filenames is not an end-to-end quality certification.

Current validation status

The validated source implementation and reproduction commands are in quality/README.md. Machine-readable evidence is in validation/2026-09-23/.

What was wrong

The previous GGUF reconstruction could produce correctly shaped matrices with scrambled values: reshaping flat GGUF storage into its fast-dimension-first dimensions and then transposing is not the correct inverse layout operation. The repair interprets the flat values directly in Hugging Face row-major shape.

The conversion also needs to reverse Qwen's zero-centered RMSNorm transform, restore A_log from the stored negative exponential, and undo the linear-attention value-head permutation. The gated linear-attention norm must not receive the zero-centered norm subtraction. These transforms now have value-level regression tests.

The Q1 sign encoding is byte-major and little-endian, with one FP16 scale for each 128 weights. That bit order was not the cause of the reconstruction failure.

Full-model source repair

The quality/ loader retains the original packed sign bits and scales rather than expanding all 26.90 billion parameters to FP32 or requantizing them to INT8. Embeddings gather only requested rows before decoding; linear layers decode bounded row chunks. Unknown, duplicate, missing or shape-mismatched tensors fail explicitly.

The initial full-source smoke run correctly answered four deterministic prompts, including arithmetic, a capital-city question and exact output. Its parameter/buffer storage was 4,216,748,288 bytes and peak CUDA allocation was 4,642,257,408 bytes on an RTX 4090. These measurements do not establish Android memory fit. The published source validator adds twelve named cases and records actual answers and token IDs; consult its report for the observed result.

The separate small LiteRT prototype passed six numerical comparisons with maximum absolute error 1.049041748046875e-05. A successful tiny graph is not evidence that the complete model exports or runs in Gallery.

Historical artifacts — unchanged

No existing weight artifact is replaced by this source-repair commit. The old experimental/ scripts are preserved for provenance, but their transpose Linears reconstruction is superseded by the strict loader in quality/.

Required before an Android release

The repaired source must pass a compatible full-model export, complete-graph LiteRT parity, tokenizer/stop-token checks, measured memory on the 8 GB x86_64 AVD, and actual generation in official Edge Gallery using the exact exported artifact hash. App installation and launch alone cannot satisfy these gates.