laguna-xs2-phone-k4-fold3-litert

Laguna XS.2 Phone K4 Fold 3 LiteRT-LM

This is a hackathon experiment that asks a direct question: can a real poolside/Laguna-XS.2 checkpoint be cut down far enough to load on an Android phone through Google's LiteRT-LM tooling?

The answer is: structurally yes, and partially behaviorally. This repository contains a real stripped Laguna XS.2 derivative, not a smoke model, LoRA, adapter, or distilled student. It is produced by MoE expert pruning, a tiny recovery tune, and LiteRT-LM dynamic int8 quantization.

The target device during the experiment was a Samsung Galaxy Fold 3 / Flip 3 class Android phone. Quality is still experimental, but this artifact is the best phone-scale checkpoint we found in the exploration.

Artifact

Use the root package:

model.litertlm

litert-lm-peek verifies that the package contains:

LlmMetadataProto
HF_Tokenizer_Zlib
TFLiteModel prefill/decode
TFLiteModel embedder

Both the quantized prefill/decode TFLite and the quantized embedder TFLite load with ai_edge_litert.interpreter.Interpreter.

What We Built

Starting from a fresh Hugging Face snapshot of poolside/Laguna-XS.2, we built a phone-scale MoE checkpoint with these steps:

1. Validated the upstream model first, to make sure the source checkpoint and chat template were healthy.
2. Pruned Laguna's routed MoE experts from 256 experts per sparse layer down to 4 retained experts per sparse layer.
3. Sliced router rows and expert correction-bias entries so the routed expert metadata matched the retained expert set.
4. Reduced the mobile runtime context/cache target to 512 tokens during the LiteRT-LM export.
5. Patched the local derived checkpoint's expert forward path into a static batched expert computation so LiteRT export avoided the aten.histc lowering path that appeared in the Transformers MoE integration.
6. Ran a tiny recovery tune on the pruned checkpoint. The final recovery pass trained only the last four transformer layers, the final norm, and the LM head for 3 epochs on a short-answer calibration set.
7. Exported the recovered checkpoint through Google's LiteRT-LM tooling.
8. Applied dynamic int8 weight quantization to the prefill/decode model and packaged the final .litertlm artifact.

This is not full distillation. The model remains a surgically reduced Laguna checkpoint, with a small post-prune recovery pass.

Testing And Results

Source Sanity Check

Before pruning, the fresh poolside/Laguna-XS.2 snapshot was run through local Transformers inference. It answered the control prompt:

Who is the president of France?
=> As of my last update, the President of France is Emmanuel Macron.

That confirmed the starting weights and template were usable.

Compression Candidates Explored

We tried several expert-reduction strategies before selecting the final K4 recovered artifact.

The biggest lesson was that blind expert deletion is very destructive for this architecture. K128 was the smallest no-training cut that showed partial semantic recovery, but it was much too large for a Fold 3 target. The K8 expert averaging path also looked attractive on paper, but in practice it destroyed generation quality.

Recovery Testing

The first K4 artifact loaded and exported, but its local HF generation collapsed into repeated malformed text on simple factual prompts. We then tested two recovery passes:

The first recovered HF checkpoint produced these local probe results:

So the result is not a general-quality assistant. It is, however, the first phone-scale K4 Laguna variant in this exploration that stopped the immediate token-loop failure on the France control prompts.

After Android testing in Google Edge Gallery showed a common failure mode where plain prompts returned only a single character and a period, such as 1., we reproduced that exact symptom locally with:

Who is the president of France?
=> 1.

We then ran a v2 recovery pass with additional prompt variants matching the mobile prompt shape. The v2 HF checkpoint produced:

This v2 checkpoint is the artifact currently packaged at the repository root. It fixes the specific one-character response seen on-device, while the broader quality warning still applies.

LiteRT-LM Export And Quantization Testing

The recovered checkpoint exported successfully through LiteRT-LM:

Expert BATCH_MATMUL ops were intentionally left unquantized because the current AI Edge quantizer failed Laguna's high-rank expert batch-matmul path. The rest of the eligible prefill/decode graph was quantized with dynamic int8 weights.

Validation performed:

litert-lm-peek: package parsed as LiteRT-LM 1.5.0
ai_edge_litert interpreter: quantized prefill/decode model loaded
ai_edge_litert interpreter: quantized embedder model loaded
Hugging Face remote SHA: verified after upload

Known Limitations

This artifact is a mobile runtime and compression research submission, not a finished assistant model.

Known weaknesses:

• It fails some held-out factual and arithmetic probes.
• It was recovered on a tiny short-answer set, so it can overfit.
• K4 retained-expert capacity is likely below the quality cliff for broad use.
• Decoding settings can reduce visible repetition but cannot restore the expert capacity removed by pruning.
• The package is intended for loading experiments in LiteRT-LM / Edge Gallery style runtimes; broad Android compatibility is still being tested.

Suggested Android Generation Settings

For Google Edge Gallery or LiteRT-LM experiments, first make sure the app is using the v2 artifact. Because the root filename is still model.litertlm, Edge Gallery may cache the previous package. Delete the old downloaded/imported model in the app, then re-download or re-import from this repository. The v2 root artifact is:

size: 3,037,564,832 bytes
sha256: 16164c9cf7d32c51096dedbdcd9a08b5b57527468764e15dba115906f9bc262c

Recommended first-test prompt:

Who is the president of France?

Expected v2-style answer:

Emmanuel Macron is the president of France.

Use short generations. This model is small and brittle, so long completions increase the chance of repetition or drift.

If Edge Gallery exposes fewer controls, use the closest available values:

max tokens: 64
temperature: 0.25
top-k: 32
top-p: 0.85

These settings help suppress repetition and avoid weak tail tokens, but they do not make the model reliable on broad knowledge or math.

Takeaways

The most useful finding is that the path to a truly good phone Laguna probably is not "delete almost all experts and hope." The best next direction is likely a larger retained-expert model with more aggressive quantization, activation- aware expert selection, and a stronger recovery objective.

Promising follow-ups:

• K16 or K32 activation-aware pruning instead of simple K4/K8 pruning.
• Mixed precision expert quantization instead of deleting nearly all routed experts.
• AWQ-style calibration to protect salient channels before low-bit export.
• Short hidden-state or expert-output reconstruction recovery, if full distillation is still out of scope.
• Better runtime decoding controls, including repetition or frequency penalties, if exposed by the Android app.

Reproducibility

The research/ folder contains the scripts and notes used for this submission, including:

research/scripts/recover_laguna_lm_head.py
research/scripts/eval_laguna_prompts.py
research/scripts/laguna_hf_native_merge_experts.py
research/experiments/k4_last4_recovery_litert_artifact.md
research/experiments/k4_last4_recovery_v2_litert_artifact.md

The older K16 org repository was removed after this recovered K4 artifact became the preferred hackathon submission.