Llama-3.2-1B-Sarcasm-Rewriter

Llama-3.2-1B-Sarcasm-Rewriter

A fine-tuned Llama-3.2-1B-Instruct that rewrites sarcastic news headlines as neutral, factual equivalents while preserving the underlying meaning.

Built by CS4248 Team 14 (NUS, AY2025/26 Semester 2) as part of a sarcasm style transfer research project.

Task

Input: A sarcastic news headline (e.g. "Area Man Passionate Defender Of What He Imagines Constitution To Be") Output: A non-sarcastic rewrite (e.g. "A man strongly defends his interpretation of the Constitution")

Training

• Base model: meta-llama/Llama-3.2-1B-Instruct
• Method: LoRA (r=16, alpha=32, dropout=0.05) targeting q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable parameters: ~11.3M (0.9% of base)
• Dataset: 8,258 sarcastic->non-sarcastic headline pairs derived from NHDSD (News Headlines Dataset for Sarcasm Detection). Non-sarcastic targets were generated by an LLM annotator (StepFun Step-3.5 Flash) informed by the original article body where available, with cross-validation by Nemotron. Split: sar_to_non_context_enhanced.
• Loss: Computed only on assistant response tokens (not on the prompt)
• After training, the LoRA adapter was merged into the base weights via merge_and_unload().

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "SeeYangZhi/Llama-3.2-1B-Sarcasm-Rewriter"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

messages = [
    {
        "role": "system",
        "content": (
            "You are a writing assistant. Rewrite sarcastic news headlines as neutral, "
            "factual equivalents that preserve the core meaning without irony or mockery. "
            "Respond with only the rewritten headline, no explanation."
        ),
    },
    {
        "role": "user",
        "content": "Rewrite this sarcastic headline as a neutral, non-sarcastic news headline:\n\narea man passionate defender of what he imagines constitution to be",
    },
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Evaluation

Evaluated on a 2,857-sample held-out test split against 13 other models (BART variants, T5 baselines, ablation studies). Metrics:

Full per-metric numbers are published alongside the project webapp.

License

This model is released under the Llama 3.2 Community License. The model name starts with "Llama-" as required by Meta's terms. Built with Llama.

Citation

If you use this model, please cite the underlying Llama 3.2 release and the NHDSD dataset. Project writeup available upon request.