tinybrainbot-303mV2-instruct

TinyBrainBot 303M V2 — Instruct

A ~303M parameter instruction-tuned language model, trained from scratch on a single 2×GPU workstation. This is the chat/instruct variant; the pretrained foundation is nkthebass/tinybrainbot-303mV2-base.

It's a genuinely small model — think GPT-2-small class — built as a from-scratch LLM project. It knows a fair amount of factual trivia, holds a short chat, greets, gives simple advice, and does basic add/subtract arithmetic with shown work. It is not a general assistant and will confidently hallucinate; see Limitations.

Architecture

Llama-family (RoPE, RMSNorm, SwiGLU, GQA), tied embeddings.

Chat format

Single-token role markers, EOS = <|end|>:

<|user|> {message} <|end|> <|assistant|>

The chat template is embedded in tokenizer_config.json (and in the GGUFs), so apply_chat_template / llama-server --jinja handle it for you.

Usage

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
msgs = [{"role": "user", "content": "What is the capital of France?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))   # -> Paris.

Tip: use greedy (do_sample=False) for factual/arithmetic queries — at this size, sampling wanders. It is also very sensitive to typos (a misspelled "captial" derails it).

llama.cpp / Jan / LM Studio

GGUF files (F16, Q8_0) are included in this repo. In llama.cpp: llama-server -m tinybrainbot-303mV2-instruct-Q8_0.gguf --jinja.

Benchmarks

Standard log-likelihood multiple-choice (lm-eval style, n=200, seed 42). Headline = acc_norm (HellaSwag/ARC/OpenBookQA), acc (WinoGrande/MMLU):

Real signal is on the QA benches (beats GPT-2-124M on ARC-Easy and beats both GPT-2-124M and Pythia-410M on ARC-Challenge). HellaSwag / MMLU / WinoGrande sit at the random floor — the size ceiling of a 303M.

Arithmetic: trained with a verified "show-your-work" math set, so it does addition and subtraction correctly with column steps (e.g. 462 + 23 → shows the ones/tens/hundreds and answers 485). Multiplication and division are still wrong — it attempts the scratchpad but the digits are off.

Training

Pretrained (see the base model) then SFT'd. SFT mix: smoltalk, a synthetic instruction set, multi-step reasoning word problems, verified arithmetic-with-work, multi-turn dialogues, and greetings. Precision fp16, WSD schedule, DDP on 2× Tesla P100.

Limitations

• It hallucinates. It's 303M — it does not reliably know facts beyond common trivia, and states wrong answers with full confidence.
• Mul/div arithmetic is broken (add/sub is fine).
• English only, 1024-token context, greedy-decoding recommended, typo-fragile.
• No safety tuning / RLHF. Do not deploy in anything user-facing without your own guardrails.

License

Apache-2.0. Free to use and build on — attribution appreciated. Trained on public/open datasets (FineWeb-Edu, Wikipedia, TinyStories, OpenWebText2, plus synthetic distillation data); please respect the licenses of those upstream sources.