writing-tools-qwen3

writing-tools-qwen3

LoRA adapter for Qwen3-4B-Instruct-2507, fine-tuned for three Apple Intelligence–style writing tasks:

• Summarise — concise paragraph summaries
• Rewrite — formal/casual tone rewrites
• Smart Reply — email replies as JSON (subject, body, tone)

Approach

QLoRA 4-bit NF4 fine-tuning (r=16, α=32) on ~3.6k SFT examples. Part of writing-tools-nlp, which also implements speculative decoding, guided generation, and MLX on-device inference.

Evaluation (val split, n=400)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-4B-Instruct-2507"
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "aditya-ice/writing-tools-qwen3")
tokenizer = AutoTokenizer.from_pretrained(base)

messages = [
    {"role": "system", "content": "You are a writing assistant. Summarise the following text in one concise paragraph."},
    {"role": "user", "content": "Your text here..."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

• Base model: Qwen/Qwen3-4B-Instruct-2507
• Method: QLoRA (peft + trl SFTTrainer)
• Data: CNN/DailyMail, PAWS, Enron emails (~4k examples)
• Hardware: Colab T4, 1 epoch