lattice-quark-1.5b

Lattice Quark

A 1.5B-parameter decoder-only language model trained from scratch by Lattice — no pretrained weights used. Pretrained on consumer hardware (RTX 5090), then instruction-tuned.

Specs

Training

• Phase 1 — base: 1,500 iters, val bits-per-byte 0.542 → 0.542 at completion.
• Phase 2 — SFT: instruction tuning, val loss 0.302 at step 465.

Checkpoints were streamed to this repo live during training (checkpoints/base/…, checkpoints/sft/…) — the raw .pt files are still here for reproducibility. The training pipeline is open source: github.com/olii-dev/nano-gpt.

Honest expectations

This is a from-scratch hobby-scale model. It holds a conversation, follows instructions, writes simple code, and has a stable identity — but it makes arithmetic errors, can be verbose, and will confidently say wrong things. Not for production. That's the point: it's a small model trained in the open, warts and all.

Observed behaviour (tested 2026-08-12)

Run it

The checkpoint uses the nanochat runner (custom architecture — no transformers config yet):

git clone https://github.com/olii-dev/nano-gpt
uv sync --extra gpu
python -m scripts.chat_cli -i sft -g quark-1.5b -s 465

The Lattice lineup

• Mini — 42M, from scratch (weights)
• Spark — 1.5B, Qwen fine-tune, the flagship (weights)
• Quark (this) — 1.5B, from scratch

More at lattice-research on Hugging Face and lattice-site-lime.vercel.app.