mr_chatterbox_model

Model Card for Mr_Chatterbox_Model

Mr. Chatterbox is a language model trained entirely from scratch on a corpus of over 28,000 Victorian-era British texts published between 1837 and 1899, drawn from a dataset made available by the British Library (https://huggingface.co/datasets/TheBritishLibrary/blbooks). The model has absolutely no pre-training inputs from after 1899 — the vocabulary and ideas are formed from its training set of nineteenth-century literature.

Mr. Chatterbox's training corpus was 28,035 books, with an estimated 2.93 billion input tokens after filtering. The model has roughly 340 million paramaters, roughly the same size as GPT-2-Medium. The difference is, of course, that unlike GPT-2, Mr. Chatterbox is pretrained entirely on historical data.

The model was trained using Andrej Karpathy's Nanochat, with two epoch of supervised fine-tuning and one small supervised fine-tuning epoch for help handling edge cases.

Mr. Chatterbox is a perfect Victorian Gentleman, he's just, you know, a chatbot. Don't tell him that though!