Leviathan-MLGRU-30M-TinyStories-Instruct-v02b

Leviathan-MLGRU-30M-TinyStories-Instruct-v0.2b

This is not a general assistant model. This is a 30M MLGRU proof model trained for the Leviathan CPU runtime. It is trained on TinyStories plus a small supervised project-specific QA mixture. This repository contains a Leviathan runtime package, not a standard Transformers checkpoint. Use the Leviathan engine.py runtime to run the model locally. For project-specific QA prompts, run with --prompt-template qa. Use --prompt-template plain only for raw continuation experiments.

Intended use

This model validates that a tiny Leviathan-trained recurrent model can learn stable project-specific QA mappings after supervised QA training. It is intended for local CPU inference experiments with engine.py from the Leviathan repository.

Dense mode is the default recommendation for this 30M model. Top-K 0.9 and 0.8 preserved tested QA matching, but no Top-K speedup is claimed.

Run

python engine.py \
  --bin leviathan_mlgru_30m_instruct_v02b.bin \
  --meta leviathan_mlgru_30m_instruct_v02b_meta.json \
  --architecture mlgru \
  --top-k 0 \
  --max-new 80 \
  --prompt-template qa

Verified QA examples

USER> What is Leviathan?
ENGINE> Leviathan is an experimental CPU inference engine for ternary and recurrent language models.

USER> What is MLGRU?
ENGINE> MLGRU is a recurrent runtime path that updates state step by step instead of computing attention maps.

USER> What is Top-K activation sparsity?
ENGINE> Top-K activation sparsity keeps the largest activations and skips smaller activations to reduce computation.

USER> What dataset was this proof model trained on?
ENGINE> The first proof model was trained on TinyStories.

Files

leviathan_mlgru_30m_instruct_v02b.bin
leviathan_mlgru_30m_instruct_v02b_meta.json
leviathan_mlgru_tokenizer/
report.json
sample_outputs_v02b.txt