murmur-360m

murmur-360m

A small model that writes haiku in one person's voice, in a browser, with no server.

SmolLM2-360M-Instruct with a LoRA fine-tune on 607 hand-curated haiku, exported to ONNX and quantised to q4f16. The download is 312 MB; it loads in about 2.5 s and generates a haiku in about 1.4 s on WebGPU.

The corpus is the point. Every example was written and then judged one at a time by a human against a written voice doctrine — 607 keeps out of 688 written, across ten batches, with the rejections and the reasons kept.

Use

import { pipeline } from '@huggingface/transformers';

const gen = await pipeline('text-generation', 'USERNAME/murmur-360m', {
  device: 'webgpu', dtype: 'q4f16',
});

// The system prompt must be exactly this string — see below.
const prompt = gen.tokenizer.apply_chat_template(
  [{ role: 'system', content: 'You write murmur haiku.' },
   { role: 'user', content: 'Topic: the office air conditioning' }],
  { tokenize: false, add_generation_prompt: true });

const out = await gen(prompt, {
  max_new_tokens: 48, do_sample: true,
  temperature: 0.3, repetition_penalty: 1.05, return_full_text: false,
});

the air conditioning in
the office is a low-grade
sulfuric acid

Three things that are not optional

The system prompt is You write murmur haiku. — byte for byte. Every training example used that exact string, and the model's distribution is conditioned on it. Six richer prompts were measured and every one reduced variety and increased the model's most over-used construction. Prompting a fine-tuned model degrades it in the same way prompting a base model improves it.

Temperature 0.3. Judged blind by the corpus author against 0.2/0.4/0.6/0.8 and greedy: the keep rate is 79% at 0.3 and 19% at 0.8. Greedy is deterministic, so it cannot support a "give me another one" button.

Prefer min_p ≈ 0.05 to top_k. Blind-judged at 70% keep against 40% for top_k=50. top_k keeps a fixed number of candidates however confident the model is, which on a 360M model leaves a junk tail permanently within reach. transformers.js has no min_p; it can be supplied as a custom logits processor.

Avoid repetition_penalty above ~1.05. At 1.15 it scores better on every monotony metric and quietly destroys topical relevance — the penalty down-weights tokens from the prompt, and the prompt is the topic.

Known limits

• It ends a clause mid-line sometimes. It learned "three lines, roughly this long, then stop" without learning to finish a thought inside that budget.
• It over-uses direct address. It opens with "coffee, I say" about 30% of the time against the corpus's 10%.
• Roughly 30% of generations are not worth keeping, judged by the person whose voice it is. Generating two or three and choosing is the intended use.

Honest provenance

A well-prompted base SmolLM2 scores about the same on "is this a good haiku". What the fine-tune buys is that the haiku sound like this person rather than like generic workshop poetry, at 312 MB instead of 570 MB and 1.4 s instead of 30 s, with no imagery in the prompt for the model to parrot back.

Trained with LoRA r=8 (5.0M trainable parameters), 4 epochs, on an Apple M-series laptop in under seven minutes.