prose-rewriter-4b-v2

prose-rewriter-4b-v2

A paragraph-level prose rewriter: it takes prose written by a large model and re-renders it to be more human, preserving the semantics it was given.

Qwen/Qwen3-4B-Base with a rank-32 LoRA merged in at strength 1.10.

Successor to prose-rewriter-4b-v1.6.

New prompt: paragraph in, rewrite out. The edit block is gone. Every training row now holds the human paragraph's own length, so there is no length mode left to name. Do not send v1.x's edit block to this checkpoint.

Markup passes through. About 15% of training rows wrap part of the paragraph in markup -- XML/HTML, BBCode, brackets, template syntax, markdown, LaTeX -- The model preserves the tags while rewriting the content.

Also fixed some bad data: targets that opened on the tail of someone else's sentence (, narration...), and AO3 sentences split around italic spans.

Both v2 releases are trained on the same pool.

Against v1.6 it edits the same amount and holds length slightly better, but keeps more of the input's slop. See Evaluation.

Variants

The quants carry the chat template and stop on <|im_end|>, and the adapted output head is kept separate from the token embeddings in both -- Q8_0 stores it at Q8_0, Q4_K_M at Q6_K.

Prompt format

<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>rewrite

The chat template in this repo builds exactly that string, byte for byte, from one message:

messages = [{"role": "source", "content": paragraph}]
tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

It is not a chat model. Every message is rendered as the source paragraph, so a runtime that probes the template with a user message still gets a valid prompt. Send one message per call.

Serving recipe

Sampled at temperature=0.9, top_p=0.9. Temperature 0.9 has been tested internally to be the most optimal value. It's recommended you use this.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chartreuse-verte/prose-rewriter-4b-v2"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

def rewrite(paragraph):
    text = tok.apply_chat_template(
        [{"role": "source", "content": paragraph}],
        tokenize=False, add_generation_prompt=True,
    )
    ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
    out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.9, top_p=0.9)
    return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()

Same thing under llama.cpp. The role is source, which no chat API models, so build the string yourself; <|im_end|> stops it:

llama-cli -m GGUF/prose-rewriter-4b-v2-Q8_0.gguf -no-cnv -n 512 --temp 0.9 --top-p 0.9 \
  -p '<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>rewrite
'

Input length

The training pool's median input is 37 words and 81% of it is under 70, so serve it on anything from a full sentence up.

The practical floor is about 15 words. Below it the failure mode is padding, cutting and fabrication rather than gibberish. Below 80 bytes, pass the text through unchanged.

Evaluation

Both releases measured as they ship -- v1.6 baked at strength 1.30, v2 at 1.10 -- on 365 held-out paragraphs of LLM-written prose that neither model saw in training, each sent its own prompt at temperature=0.9, top_p=0.95, three swipes each. 1,095 generations an arm.

Paired over the 303 inputs of 51-90 words, v2 preserves more length (t = +3.1) and runs past 1.2x the input's length slightly more often (0.3% to 1.5%, t = +2.9). Every other paired t falls between -1.5 and +1.7.

Unsupported content -- a sentence in the rewrite the input does not entail, scored by NLI with the input alone as the premise:

On the same 303 inputs neither separates: unsupported t = +1.9, coverage t = +1.1.

The register numbers on the same paragraphs, against the input:

Paired over all 365 inputs, all three rise: banned constructions t = +4.5, density t = +4.9, purple t = +2.5. The extra banned constructions are carried over from the input, not introduced: 236 retained against v1.6's 156, 23 introduced against 16.

The v1.6 column was measured alongside v2 for this table and does not match v1.6's own card, which compared against v1.4 at top_p=0.9.

A note on strength

Every number above is measured at the strength the release is baked at, which is not the same as measuring the adapter. LoRA strength is spent at merge time (W + (B @ A) * 2.20 here), and the metrics move with it: read at the adapter's natural 1.0 this same checkpoint is a materially different artifact.

1.10 is the lowest strength at which the pass-through and near-verbatim rates bottomed out on the strength sweep.

Training

Corrupt forward, train backward. The human paragraph is the target; an on-policy LLM manufactures the input by slop-ifying it.

The target side is human prose: r/WritingPrompts (Mollymo/Human-to-AI-writing), AO3 (midwestern-simulation-active/ao3_random_subset), a scrape of bluemoonroleplaying.com -- the only human writing in the pool already in the deployment's own register -- and a sliver of fanfiction.net (atom-in-the-universe/fanfics-10k-10k).

Professionally edited fiction. Every source above is amateur archive prose. Published contemporary literary fiction now supplies about a third of the target side -- the first professionally edited register in the pool, added because it is the one class that clears the corruptor's own output by a margin wide enough to register on vocabulary breadth.

The input side is manufactured from those targets, weighted and share-capped so that no single generator's tics dominate:

Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the scene, grammatical correctness on the target side, content recall stratified by target length, and NLI entailment both ways. Two further screens shape what reaches training -- a floor on how much a pair actually changes, and a floor on the sentence-length variety of the target, applied at a higher threshold for long paragraphs than short ones.

Loss on the target paragraph only. Everything before rewrite is masked.

The merge

Merged at strength 1.10. Rank 32 with alpha 64 is a LoRA scaling of 2.0, so the effective scaling is 2.20: W + (B @ A) * 2.20. Merged in float32, stored bfloat16.

lm_head is adapted, and Qwen3-4B-Base ties lm_head.weight to embed_tokens.weight. This checkpoint is untied: the merged output head is stored separately and the input embeddings are bit-identical to the base model's, which is what training assumed. config.json says tie_word_embeddings: false and it means it. Do not re-tie it, and if you convert to another format, check that the head survived.

Limitations

• Not an instruct model. It has one job and one prompt. There is nothing to ask it.
• Works on fictional prose only. May not work on technical documentation.
• One paragraph per call. Longer input degrades; split it.
• Keeps more of the input's slop than v1.6. 3.30 banned constructions per 1k words against 2.22, nearly all of them carried over from the input.
• Truncates. It drops below 0.75x the input's length on 16.2% of paragraphs, level with v1.6, and shortens single sentences more often than v1.6 did. Check the output when length matters.
• Will not pass AI detectors. Pangram and such will still know because this model preserves word choices and certain sentence structures.
• English only, narrative register (third and first person fiction, dialogue with quoted speech).
• Short input pads, cuts and invents. The floor is about 15 words. See Input length.
• Repeats a noun sooner than an LLM would. Human prose reuses a plain noun where generated prose uses a synonym, and this model has learned that habit.

License

The weights in this repository are released under the GNU Affero General Public License, version 3. The full text is in LICENSE.

This is a derivative of Qwen/Qwen3-4B-Base, which is licensed under Apache License 2.0. That license is preserved and its terms continue to apply to the base weights this model was built from; the AGPL covers the combined work as distributed here. Apache-2.0 is one-way compatible with AGPLv3, which is what makes this combination possible.

If you run a modified version of this model as a network service, AGPL section 13 requires you to offer the corresponding source of your modifications to its users.