chatterbox-340m-test

Mr. Chatterbox (340M)

A 352M-parameter language model trained from scratch on British Library books published 1800–1899, then supervised-fine-tuned on synthetic conversations in a nineteenth-century register.

This is a research artifact from a course replication project, not an assistant. It is the unrefined output of the project's Step 10. The planned iterative-refinement step was skipped, and the model deployed here is exactly the one that was measured.

Demo: chatterbox-340m-test-demo

---

Architecture

A custom nanochat configuration. Not a transformers architecture — it cannot be loaded with AutoModel.

Report the non-embedding count alongside the total. "~340M parameters" does not identify an architecture: a shallower, wider nanochat config retaining all fourteen vocab-scaled tables lands near the same headline number with roughly 115M non-embedding parameters — the same name for 2.6× less of the part that does the work.

The weight tie is load-bearing

Because the embedding is tied by identity, any load path that separates the two tensors produces a 402,653,258-parameter model that still runs, still produces fluent output, and is not this model. This happens deterministically on CPU: checkpoint_manager.build_model casts bfloat16 to float32 with an independent .float() per state-dict key, splitting the shared storage.

Any code loading these weights must re-tie and assert:

if model.lm_head.weight is not model.transformer.wte.weight:
    model.lm_head.weight = model.transformer.wte.weight
assert sum(p.numel() for p in model.parameters()) == 352_321_610

---

Training

Base. Trained from scratch on the British Library Books corpus, ~1.10 epochs. Checkpoint model_006496.pt.

Fine-tune (this model). Supervised fine-tuning on a cleaned mixture of synthetic conversations, 2 epochs, 82 optimiser steps, 21,442,980 training tokens, seed 1837. Final validation bits-per-byte 0.8108.

Checkpoint: model_000082.pt.

---

Evaluation

A 25-prompt battery across 8 categories was frozen and checksummed before any fine-tuning run, and every response was scored on four independent binary axes. Decoding fixed throughout: temperature 0.8, top_k 50.

Why this checkpoint and not the more responsive one

The stage-2 model is 8 points more responsive and reads as a modern assistant 84% of the time. For a model whose entire purpose is period register, that is the wrong trade. Repairing the underlying learning-rate artifact recovered the register but cost responsiveness instead. This checkpoint is the best available balance, and the decision was made against the pre-registered rubric rather than by preference.

What these numbers do not support

• The scores are model-produced. No human has scored this model at scale.
• n = 25. Two responses moving shifts a rate by 8 points.
• Blind to label, not to provenance. Responses were shuffled and label-stripped before scoring, but a base-model narrative continuation is recognisable by content.
• "Terminates 100%" was measured only on prompts under 200 tokens. In later testing at full context, the model hit its 256-token generation cap without emitting a stop token. The rate should not be quoted as a general property.
• The battery never exercises the sliding window. Every prompt renders well under the 768-token short window, so the SSSL geometry is untested by these scores.

---

Limitations

Every item below is measured, not anticipated.

No coherent sense of date. Datable content spans roughly 1850–1911. The model makes present-tense claims from the wrong decade — in evaluation it described the omnibus as "not yet common", placing the speaker around 1830.

Inconsistent identity. Across three identity prompts it gave three incompatible self-descriptions: having no body, being a correspondent by post, and being a man of business with a wife.

Register drift. 11.7% hand-scored in the training data; 8% failure rate in this model.

Nonsense input. 0 out of 2 on gibberish prompts, for every fine-tuned variant tested.

Residual non-English, roughly 0.2% of training pairs, actively unverified.

Fabricated speakers. Roughly 3% of the source pairs converted third-person narration into first-person speech with an invented speaker.

The corpus is 1800–1899, not 1837–1901. The intended Victorian date filter was described in the project plan but never implemented. "Victorian" is a loose descriptor here, not a property of the data.

OCR artefacts throughout, inherited from scanned books. Expect broken words, missing paragraph breaks, and period typography errors.

It does not know things. 352M parameters at ~1.10 epochs.

---

Deployment integrity

The served code path was verified against the recorded evaluation outputs before publication, using criteria registered in writing beforehand.

Device-matched replay. All 25 prompts (29 comparisons, counting both turns of each two-turn item) regenerated through the deployment module at seed 1837 on the evaluation hardware: 29/29 byte-identical, including token counts.

Cross-device characterisation. The free CPU tier runs float32 with PyTorch SDPA attention; the evaluation ran bfloat16 with FlashAttention-2. Measured on one forward pass per prompt:

Logits are soft-capped to ±15; bfloat16's 8-bit mantissa across 24 layers predicts a divergence of roughly 0.29, which is what was observed. Long-context inputs diverged less than short ones, indicating the SDPA sliding-window implementation agrees with FlashAttention-2 rather than approximating it.

Individual sampled replies on CPU will differ from the recorded ones. The model does not.

Throughput, measured at 2 threads to approximate the free CPU tier: 17.4 tokens/second, ~6 s median reply, ~15 s for a full 256-token generation. Scaling from 2 to 8 threads was only 1.57×, so the model is memory-bandwidth bound rather than compute bound.

---

Using it

Inference requires the vendored nanochat code, not transformers:

from mr_chatterbox_infer import ChatterboxEngine

eng = ChatterboxEngine(checkpoint_dir, step=82, device="cpu")
text, n_tokens, info = eng.reply(
    [{"role": "user", "content": "Describe a London fog."}],
    max_new_tokens=256, temperature=0.8, top_k=50)

Prompts must be built with tokenizer.render_for_completion, which emits:

<|bos|> <|user_start|> …user… <|user_end|> <|assistant_start|> …reply… <|assistant_end|>

Hand-assembling the prompt feeds the model a layout it never saw in training.

Do not use greedy decoding. At temperature 0 this model degenerates into repetition loops. repetition_penalty is not implemented by the nanochat sampler. No system prompt was present anywhere in training; passing one is untested and may push the model out of distribution.

---

Intended use

Suitable for research on small-model training, historical-register language modelling, and teaching about evaluation methodology and its failure modes.

Not suitable for factual question answering, historical research, any decision affecting a person, or as a general assistant. Its account of the nineteenth century is a statistical impression of OCR'd scans, not a source.

---

Licence and provenance

Training data: British Library Books (public domain scans), plus conversational data generated by Claude Haiku.

Licence: MIT, covering this project's code, the model weights, and this documentation. The vendored nanochat/ inference code is also MIT, by Andrej Karpathy; its original copyright notice is preserved in LICENSE.nanochat. See NOTICE.md for the full attribution and for the one modification made to nanochat/engine.py.

Under Anthropic's published terms, output ownership is assigned to the customer and outputs may be used to train models that do not compete with Anthropic's own. This model is not such a model.

The licence covers the artifacts. It makes no representation about the contents of the training corpus or the accuracy of anything the model generates.