NanoChat-28M-V1

NanoChat-28M

A ~27.8M parameter GPT-style decoder-only transformer, trained fully from scratch, that generates simulated live-stream chat comments -- both regular fast-paced "hype chat" and rarer, longer "superchat"-style paid messages/questions. Built to run fast on CPU, as a lightweight audience-simulation layer for AI VTuber / livestream projects.

Model details

• Architecture: custom GPT-style decoder-only transformer (not a registered HF architecture -- see model.py in this repo for the class definition)
• Parameters: 27,784,960 (~27.8M)
• Layers: 10 | Hidden size: 448 | Attention heads: 8 | FFN size: 1792
• Context length: 128 tokens
• Tokenizer: byte-level BPE, vocab size 8,000, trained on the same chat corpus
• Weight tying: input/output embeddings are tied
• Inference: supports KV-caching for fast autoregressive generation (see model.py's generate()) -- ~36 generations/sec, ~920 tokens/sec at batch_size=32 on a consumer CPU (fp32; int8 dynamic quantization was tested and found to not help once KV-caching is used, at this model scale -- fp32 is the recommended path)

Two generation modes

The model was trained with mode-marker tokens so it can be conditioned to generate either style:

• <chat> -- short, high-volume, low-effort hype comments ("LMAOOO", "W stream", emote spam), matching typical live chat cadence.
• <superchat> -- rarer, longer, more coherent messages: questions, personal questions, playful roasts, jokes, and support messages, matching the style of paid superchat/membership messages that streamers are expected to acknowledge.

Usage

import torch
from tokenizers import Tokenizer
from safetensors.torch import load_model
from model import ChatGPTMini, ModelConfig  # model.py included in this repo

tokenizer = Tokenizer.from_file("tokenizer.json")
cfg = ModelConfig(vocab_size=8000, pad_token_id=tokenizer.token_to_id("<pad>"))
model = ChatGPTMini(cfg)
load_model(model, "model.safetensors")
model.eval()

bos_id = tokenizer.token_to_id("<bos>")
chat_id = tokenizer.token_to_id("<chat>")
eos_id = tokenizer.token_to_id("<eos>")

prompt = torch.tensor([[bos_id, chat_id]])
out = model.generate(prompt, max_new_tokens=30, temperature=0.9, top_k=40, top_p=0.9, eos_token_id=eos_id)
print(tokenizer.decode(out[0].tolist(), skip_special_tokens=True))

See example.py in this repo for a complete runnable script, including batched generation (recommended for real usage -- throughput scales far better than looping one generation at a time).

Training data

• Regular chat: primarily real, public Twitch chat logs from the lparkourer10/twitch_chat dataset (CC-BY-SA-4.0, usernames stripped), supplemented with a small amount of synthetic hype-phrase data.
• Superchat: hand-written and template-generated, since no public dataset of real superchat/membership messages exists. This class is intentionally class-imbalanced relative to chat volume (~700 unique examples vs. millions of chat rows) and was oversampled during training via weighted sampling, not duplication.

Intended use

Generating background "audience chat" text for livestream/VTuber simulation projects, demos, or similar creative/entertainment use cases where large volumes of low-effort, stylistically plausible chat text are needed cheaply and quickly. Not intended as a factual, instruction-following, or general-purpose assistant model -- it has no knowledge base beyond chat-style text patterns.

Limitations

• Small model, narrow training objective -- will occasionally produce garbled or nonsensical text, especially at higher sampling temperatures. This is generally visually consistent with how noisy real chat already looks, but shouldn't be mistaken for a general-purpose language model.
• Superchat generation may lean close to its (relatively small) training examples rather than fully generalizing, given the limited amount of real-world superchat data available to train on.
• No built-in context/topic conditioning -- the model only knows "chat mode" vs. "superchat mode." Topical relevance to a live stream is intended to be handled externally (e.g. by seeding the generation prompt with a keyword extracted from a transcript), not by the model itself.

License

Released under CC-BY-SA-4.0, matching the license of the primary training dataset.