llm-of-life-lora

LLM of Life, LoRA adapters

LoRA adapters on Qwen2.5-0.5B-Instruct that turn the model into a Game of Life update rule, for llm-life's browser demo.

Try the demo or the compare page.

What these are

llm-life runs Conway's Life as the update rule of several small models instead of the classical rule. Two of the methods are LoRA adapters on top of Qwen2.5-0.5B-Instruct Q4_0 (base weights not included here, see Usage): variant A reads one cell's 3x3 neighbourhood per forward pass, variant B reads the whole grid in one forward pass with a stencil-masked attention pattern. The three from-scratch comparison models (a tiny BERT, an MLP and a stencil attention model) live in idle-intelligence/stencil-life.

Files

All four LoRA files are loaded as runtime deltas onto the Q4_0 base model already resident in the browser (llm_wasm::lora), not as offline GGUF merges.

Usage

Loaded by llm-life's demo page (web/index.html, web/compare/index.html) via the Burn/WebGPU engine from llm-web (llm_wasm), compiled to WASM. The base model and tokenizer are fetched directly from Qwen/Qwen2.5-0.5B-Instruct-GGUF and Qwen/Qwen2.5-0.5B-Instruct (CORS-enabled, not re-hosted here); this repo supplies only the adapters.

• lora-a-norules-300.bin / lora-a-rules-300.bin: variant A, per-cell mode ("LLM per cell" in the demo).
• lora-b-16-s3-300.bin (at 16x16) / lora-b-32.bin (at 32x32): variant B, whole-grid mode. No adapter is trained at 64x64, so that grid size is unavailable in this mode.

Training

All variant A/B LoRA runs: rank 8, alpha 16, applied to q/k/v/o, trained against Qwen2.5-0.5B-Instruct Q4_0.

• lora-a-norules-300.bin: 80 steps, batch 64, lr 3e-5, seed 1, trainable parameters 1,081,344, no rule text in the prompt prefix (docs/runs/2026-09-20-ft-a-norules-300.md).
• lora-a-rules-300.bin: 100 steps, batch 64, lr 3e-5, seed 1, trainable parameters 1,081,344, rules stated in the prompt prefix (docs/runs/2026-09-20-ft-a-rules-300.md).
• lora-b-16-s3-300.bin: 16x16 torus grid, rule B3/S23, 300 steps, lr 5e-5, seed 3, trainable parameters 1,081,344 (docs/runs/2026-09-20-ft-b-16-s3-300.md).
• lora-b-32.bin: 32x32 grid, 300 steps, lr 5e-5, rank 8, alpha 16 (docs/runs/2026-09-20-b-rollout.md); the training log for this run was not kept, so only the re-scored rollout in docs/runs/2026-09-20-b-rollout.md is documented.

Results

Held-out results from the run docs above: neighbourhoods or cells correct, and whether every generation of a rollout came out exact.

For context, the un-tuned base model gets 215 of 512 neighbourhoods right with the rules stated in the prompt, and 195 of 512 without them (docs/runs/2026-09-20-base-vs-finetuned.md).

That is, from a random start, it doesn't destroy everything, but it gets much of the rules wrong. Trying to make it reproduce a glider fails 100% of the time. The full grid version immediately kills all cells at iteration 1.

Model Details

• Base model (LoRA adapters only): Qwen/Qwen2.5-0.5B-Instruct, Q4_0 quantized GGUF from Qwen/Qwen2.5-0.5B-Instruct-GGUF
• LoRA config: rank 8, alpha 16, applied to q/k/v/o, 1,081,344 trainable parameters
• Engine: llm-web's Burn 0.20 + custom WGSL WebGPU backend
• License: MIT (llm-life engine and demo); adapters here are Apache-2.0, matching the base model

Citation

@misc{qwen2.5,
    title = {Qwen2.5: A Party of Foundation Models},
    url = {https://qwenlm.github.io/blog/qwen2.5/},
    author = {Qwen Team},
    month = {September},
    year = {2024}
}

Disclaimer

This is independent work by ilnmtlbnm@idle-intelligence, not affiliated with or endorsed by the Qwen team.