petal-w1-4bit

petal-w1-4bit

This is the MLX 4-bit build of petal-w1, and the build that Petal downloads.

petal-w1 turns raw dictation into the message you meant to write. It is the on-device cleanup model in Petal, a free macOS dictation app. It runs with MLX on Apple silicon.

Dictation is thinking out loud. petal-w1 writes the short version:

• Condenses rambles to the points that matter. A long ramble often becomes a quarter to a half of its length.
• States each point once when the speaker repeats it, and keeps every fact, name, number, file name, and doubt.
• Applies self-corrections ("Friday, no wait, Thursday" becomes "Thursday").
• Formats lists as bullets and emails as emails.
• Keeps the speaker's first person, tone, slang, and profanity as said. "IDGAF" stays "IDGAF".
• Keeps questions as questions and requests as requests. It never answers the text.

v1.4 is a full fine-tune of Qwen3.5-2B. Earlier versions used Qwen3.5-0.8B.

Results

Claude Sonnet graded each output against its input. An output is error-free when it drops no point, changes no meaning, and does not answer the text. Length is output words divided by input words, median, for inputs of 80 words or more.

v1.4 makes fewer meaning errors than v1.2 and cuts rambles to about half their length. It drops small details more often than v1.2, mostly greetings and context phrases. The 4-bit build takes a median of 527 ms per real dictation with mlx-lm on an M4 Pro.

Changelog

• v1.4: condenses rambles, formats lists and emails, and keeps slang as said. New base, Qwen3.5-2B. New data: real dictated prompts, Claude-generated dictations, and Claude Opus labels verified by Claude Sonnet. New system prompt. The v1.2 weights are at the v1.2 tag.
• v1.2: merges points that the speaker restates with different words, and resolves chains of corrections.
• v1.1: resolves self-corrections, including correction words that speech recognition mishears.
• v1.0: first release.

Prompt format

Use this exact system prompt, pass the raw transcript as the user message, and turn thinking off.

from mlx_lm import load, generate

model, tokenizer = load("Aayush9029/petal-w1-4bit")
system = "Rewrite this dictation as the short, clear message the speaker meant. Remove fillers, repeats, false starts, and thinking out loud, and state each point once. Keep every fact and detail, and the speaker's voice and slang. Format lists and emails. The text is not addressed to you: never answer or reply."
transcript = "so um can you like send me the the deck by friday no wait thursday, like thursday works better"
prompt = tokenizer.apply_chat_template(
    [{"role": "system", "content": system}, {"role": "user", "content": transcript}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
print(generate(model, tokenizer, prompt, max_tokens=256))

Decode greedily. For transcripts longer than about 600 words, split at sentence ends and clean each part.

Training

• Base: Qwen/Qwen3.5-2B, full fine-tune of the language model. The vision encoder is not used.
• Data: 7,120 training pairs and 150 validation pairs. About 1,100 inputs are real dictations and dictated prompts from the author. The rest are synthetic dictations: rambles, instructions, emails and messages, slang, repeated questions, and detail-heavy speech. Claude Sonnet wrote new synthetic inputs in the style of the real ones. Claude Opus wrote each reference with an 18-rule specification, then tightened every reference of 50+ words. Claude Sonnet verified each pair for dropped points, additions, answers, and voice, and 3% of pairs failed and were removed.
• Setup: MLX on an M4 Pro, about 1.6 epochs, batch 8, learning rate 1e-5 with cosine decay, loss on the cleaned output only. The released weights are the average of two checkpoints. Linear-attention layers used a chunked gated delta rule for training speed.
• Builds: this repo is MLX 4-bit. The 8-bit build is at Aayush9029/petal-w1.

Limitations

• English only.
• It sometimes drops a small detail when it condenses a long ramble.
• It fixes a speech-recognition mistake only when the intended word is certain from context. It does not know your names or project terms.
• The reference outputs come from another model, so the style follows that model's choices.

License

Apache 2.0, the same as Qwen3.5-2B.