Couplet-base

<div align="center">

Couplet-base

Pretrained base model for the Couplet family (~268M)

A larger TinyGDN hybrid language model: recurrent GDN-2 memory plus gated GQA

</div>

---

What this is

Couplet-base is the pretrained (base) checkpoint for Couplet, the ~268M successor to the Monostich-2 family.

• Scales kerzgrr/Monostich-2-base from ~149M to ~268M parameters
• Hybrid Gated DeltaNet-2 recurrent layers + gated GQA full-attention layers
• This repo is pretrain-only raw text continuation
• Instruction-tuned / chat (SFT): kerzgrr/Couplet

If you want chat behaviour, use kerzgrr/Couplet — this base model is for continuation / research and will not follow instructions reliably.

---

Model Architecture

Pipeline: Text Prompt → BPE-49K Tokenizer → TinyGDN Hybrid Decoder (28L) → Next-token Prediction

Hybrid block schedule (×28)

Every 4th layer is full attention; the rest are Gated DeltaNet-2:

GDN2, GDN2, GDN2, GQA, … (3:1 recurrent-to-attention)

Technical specifications

---

Training (pretrain)

---

Install

1) System requirements

• Python 3.10+
• CUDA GPU strongly recommended
• PyTorch with CUDA matching your driver

2) Create an environment

python -m venv .venv
# Windows
.venv\Scripts\activate
# Linux / macOS
source .venv/bin/activate

3) Install PyTorch

Pick the build for your platform from https://pytorch.org. Example:

pip install torch --index-url https://download.pytorch.org/whl/cu124

CPU-only:

pip install torch

4) Install Python deps

pip install safetensors tokenizers huggingface_hub

Flash Linear Attention is installed automatically by inference.py on first run (pinned commit + Windows import patches when needed). Git must be on PATH.

5) Download the inference script

curl -L -o inference.py https://huggingface.co/kerzgrr/Couplet-base/resolve/main/inference.py

# or Hugging Face CLI
hf download kerzgrr/Couplet-base inference.py --local-dir .

The script auto-downloads model.safetensors, config.json, tokenizer.json, and the tiny_gdn/ package from this repo.

---

Quick start

Single prompt (streams tokens):

python inference.py --prompt "The history of computing begins"

Interactive REPL:

python inference.py

Common options:

---

Pretrain vs chat

---

Files

kerzgrr/Couplet-base/
  README.md
  inference.py
  requirements.txt
  model.safetensors
  config.json
  tokenizer.json
  tokenizer_config.json
  special_tokens_map.json
  special_token_ids.json
  merges.txt
  vocab.json
  tiny_gdn/
    __init__.py
    config.py
    model.py

---

Limitations

• Base model: not instruction-tuned; may ramble or fail at Q&A format
• Scale: ~268M parameters — research / edge prototype, not a frontier model
• Dependency: requires flash-linear-attention; not GGUF / llama.cpp compatible today
• Context: trained at 2,048; longer windows are experimental

---

Model family

---

Citation

@misc{coupletbase2026,
  title={Couplet-base: A 268M Hybrid GDN-2 + GQA Language Model},
  author={kerzgrr},
  year={2026},
  url={https://huggingface.co/kerzgrr/Couplet-base}
}

---

Acknowledgments

• flash-linear-attention (Gated DeltaNet-2)
• FineWeb-Edu
• Monostich-2 family: kerzgrr/Monostich-2
• PyTorch SDPA / Hugging Face Hub + tokenizers

---

<div align="center">

A couplet is a pair of lines — compact, connected, and complete.

</div>