bobjones-ministral-14b

bobjones-ministral-14b

Ministral 3 14B Base, continued-pretrained on the complete blog archive of Sir Bob Jones (nopunchespulled.com) and then instruction-tuned to write a column from a one-line brief.

Usage

The model expects a brief and returns TITLE\n\nARTICLE. The chat template is a minimal Mistral [INST] format shipped with the tokenizer — load the tokenizer from this repo, not from the base model, or the format is lost and output degrades badly.

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    "ariedotcodotnz/bobjones-ministral-14b",
    max_seq_length = 4096,
    load_in_4bit = True,      # ~9 GB; bf16 needs ~28 GB and will not fit a 24 GB card
)
FastModel.for_inference(model)
tok = getattr(tokenizer, "tokenizer", tokenizer)

brief = "Any thoughts on the state of headline writing in the papers lately?"
text = tok.apply_chat_template([{"role": "user", "content": brief}],
                               tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt", add_special_tokens=False).to("cuda")
out = model.generate(**inputs, max_new_tokens=800, temperature=0.9,
                     min_p=0.05, do_sample=True)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Rendered prompt format:

<s>[INST]{brief}[/INST]{TITLE}

{article}</s>

Notes. The architecture is Mistral3ForConditionalGeneration — a multimodal checkpoint with a Pixtral vision tower. The vision layers were not trained and are unused; treat this as a text-only model. min_p=0.05 with temperature=0.9 produced the best samples in testing.

Training data

1,573 posts scraped from nopunchespulled.com, 27 Feb 2019 – 16 Apr 2025. After cleaning, 1,448 articles / ~550k tokens.

125 posts were dropped: 119 were cartoon or letter-screenshot posts consisting of a title and an image the scrape never captured, plus 6 fragments. Short posts were deliberately kept — 67 articles run 8–45 words and carry the punchy one-line style.

Cleaning was intentionally minimal: NFC normalisation, CRLF → LF, non-breaking and zero-width characters removed, runs of 3+ newlines collapsed. No spelling correction, no smart-quote replacement, no sentence rewriting — the idiosyncrasies are the training signal.

Splits are at article level with a fixed seed (3407): 1,258 train / 70 validation / 70 test, plus a chronological holdout of the 50 most recent posts. Zero exact duplicates; the 2 near-duplicate pairs found were forced into the same split.

Licensing of the training data

The weights are released under Apache 2.0, inherited from Ministral 3 Base. The training corpus is copyrighted material belonging to its author and was not licensed for this use. Anyone redistributing or deploying this model should consider their own position on that; inclusion here is not a claim of rights over the underlying text.

Framework

Trained with Unsloth and TRL. Base model: mistralai/Ministral-3-14B-Base-2512 (Apache 2.0).