Leviathan-MLGRU-30M-TinyStories

Leviathan-MLGRU-30M-TinyStories-v0.1

This is a small proof model for the Leviathan-1.58bit-Engine.

It was trained to validate the route:

Modal T4 training -> Leviathan MLGRU checkpoint -> ternary 2-bit export -> local CPU runtime

This is not a general assistant model. It is a TinyStories-style recurrent language model intended for runtime validation and benchmark experiments.

Files

leviathan_mlgru_30m_t4.bin
leviathan_mlgru_30m_t4_meta.json
leviathan_mlgru_tokenizer/
report.json
sample_outputs.txt

Model details

Architecture: Leviathan MLGRU
Dataset: TinyStories
Steps: 2000
Hidden size: 512
Layers: 8
Intermediate size: 1536
Sequence length: 96
Batch size: 16
Linear weights: ternary_2bit_packed
Structural tensors: FP16
Runtime: Leviathan CPU engine

Usage

Clone or download this model repository, then run from the directory containing the model files:

python engine.py \
  --bin leviathan_mlgru_30m_t4.bin \
  --meta leviathan_mlgru_30m_t4_meta.json \
  --architecture mlgru \
  --top-k 0 \
  --max-new 120 \
  --prompt-template plain

Recommended prompt:

Once upon a time

Benchmark summary

Observed local CPU runtime:

At 30M scale, dense mode is faster because Top-K selection overhead dominates. Ratio Top-K remains useful as a sparsity-quality and scaling experiment.

Limitations

• Not instruction tuned.
• Not a factual QA model.
• TinyStories-like output only.
• Small proof model; not intended for production use.
• Top-K does not yet beat dense mode at 30M scale.

Intended use

• Leviathan runtime validation.
• CPU ternary inference testing.
• MLGRU recurrent path testing.
• Top-K sparsity experiments.
• Reproducible small-model benchmark baseline.