localgrammar-qwen25-3b-lora-v1

localgrammar-qwen25-3b-lora-v1

localgrammar-qwen25-3b-lora-v1 is a LoRA adapter for unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit specialized for local English rewriting.

It was fine-tuned for four controlled rewrite tasks:

There is no separate TASK_PHRASING tag. If you want a general phrasing rewrite, use:

• TASK_CLARITY for sentence-level rewording
• TASK_COHERENCE for flow and structure improvements

This repository contains the adapter weights and tokenizer assets. It is not a standalone base model.

Intended Use

This adapter is intended for:

• local grammar correction
• plain-English simplification
• clarity-focused rewrites
• coherence and flow edits
• controlled rewriting in offline or local-first writing tools

It is not intended as a general-purpose open-ended chat assistant.

Prompt Format

The adapter expects the task to be stated explicitly in the user message.

During fine-tuning, each example used:

• system prompt: You are a precise local writing engine. Follow the requested task exactly. Preserve the user's meaning.
• user prompt format:

LANG_EN | TASK_...
Instruction: ...
Input: ...

Example for grammar correction:

LANG_EN | TASK_GRAMMAR
Instruction: Fix grammar, spelling, punctuation, and agreement errors with minimal edits. Preserve meaning and style.
Input: She go to school yesterday.

Quick Start

Load this adapter on top of the base model with PEFT.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit"
adapter_id = "johan237/localgrammar-qwen25-3b-lora-v1"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    trust_remote_code=True,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are a precise local writing engine. "
            "Follow the requested task exactly. Preserve the user's meaning."
        ),
    },
    {
        "role": "user",
        "content": (
            "LANG_EN | TASK_GRAMMAR\n"
            "Instruction: Fix grammar, spelling, punctuation, and agreement "
            "errors with minimal edits. Preserve meaning and style.\n"
            "Input: She go to school yesterday."
        ),
    },
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
        temperature=0.0,
    )

generated_ids = output_ids[0][inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(generated_ids, skip_special_tokens=True))

Expected output:

She went to school yesterday.

How It Was Fine-Tuned

This adapter was fine-tuned with supervised fine-tuning on a chat-formatted English rewrite dataset assembled for local writing assistance.

Training data construction

The training set was built from:

• grammarly/coedit
• jhu-clsp/jfleg
• facebook/asset
• chaojiang06/wiki_auto

The dataset compiler:

• mapped source tasks into the four task tags above
• removed duplicate source/target pairs within each task
• stripped instruction-like prefixes from some source examples
• filtered obviously broken rows
• filtered very short examples
• filtered examples with large source/target length mismatches

Training set summary

• total rows: 42,247
• train rows: 42,035
• validation rows: 2,212

Task distribution:

• TASK_GRAMMAR: 20,000
• TASK_SIMPLIFY: 15,000
• TASK_COHERENCE: 8,000
• TASK_CLARITY: 1,247

Dataset usage:

• CoEdIT provided the main multi-task supervision
• JFLEG augmented grammar correction
• ASSET and WikiAuto augmented simplification

LoRA configuration

From adapter_config.json:

• LoRA rank r=16
• lora_alpha=32
• lora_dropout=0
• target modules:
  • q_proj
  • k_proj
  • v_proj
  • o_proj
  • gate_proj
  • up_proj
  • down_proj

Recorded training state

From the saved trainer state:

• supervised fine-tuning
• 1 epoch
• 1855 total training steps
• evaluation every 250 steps
• save every 250 steps
• recorded final validation loss: 0.4007
• recorded train batch size: 2

Framework Versions

The exported training metadata reported:

• PEFT 0.19.1
• TRL 0.24.0
• Transformers 5.5.0
• PyTorch 2.5.1+cu121
• Datasets 4.3.0
• Tokenizers 0.22.2

Limitations

• English only
• optimized for short-to-medium rewrite requests rather than open-ended chat
• output quality depends strongly on using the correct task tag
• some training examples come from public rewrite datasets and may retain occasional dataset artifacts
• TASK_CLARITY has much less training data than grammar or simplification

License

The base model Qwen/Qwen2.5-3B-Instruct is published on Hugging Face under the qwen-research license. This adapter should be used and redistributed in compliance with the upstream base-model license and the terms of the source datasets.

Sources

• Base model family: Qwen/Qwen2.5-3B-Instruct
• Quantized training base: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
• Datasets:
  • grammarly/coedit
  • jhu-clsp/jfleg
  • facebook/asset
  • chaojiang06/wiki_auto

Citation

If you use this adapter, cite the base model, the source datasets, and your own project/repository in addition to standard library citations.

TRL citation:

@misc{vonwerra2022trl,
  title        = {TRL: Transformer Reinforcement Learning},
  author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
  year         = {2020},
  journal      = {GitHub repository},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/huggingface/trl}}
}