BART-Base-RL-Sarcasm-Rewriter

BART-Base-RL (REINFORCE)

BART-base further trained with REINFORCE + KL penalty on top of the supervised baseline. The reward encourages high semantic similarity combined with a low sarcasm-classifier score on the output.

Part of the Project LLMao sarcasm style transfer suite (CS4248 Team 14, NUS AY2025/26 S2). This model rewrites sarcastic news headlines as neutral, factual equivalents while preserving the underlying meaning.

Task

Input: A sarcastic news headline Output: A non-sarcastic rewrite

Example:

• In: "Area Man Passionate Defender Of What He Imagines Constitution To Be"
• Out: "Man defends his interpretation of the Constitution."

Training

• Base model: facebook/bart-base (139M params)
• Method: Stage 1: SFT on the original sar-to-non pairs (10,868 train examples). Stage 2: REINFORCE policy-gradient fine-tuning with KL divergence against the SFT model to prevent drift. Reward = 0.5 * (1 - P(sarcastic)) + 0.5 * ROUGE-L.
• Dataset: 10,868 sarcastic->non-sarcastic headline pairs derived from NHDSD (News Headlines Dataset for Sarcasm Detection). Non-sarcastic targets were generated by an LLM annotator (StepFun Step-3.5 Flash) with cross-validation by Nemotron. Split: sar_to_non (original).
• Input format: Raw sarcastic headline (no task prefix — BART is not pretrained with prefixes).
• Generation: beam search with num_beams=4, max_length=128.

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = "SeeYangZhi/BART-Base-RL-Sarcasm-Rewriter"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

headline = "Area Man Passionate Defender Of What He Imagines Constitution To Be"
inputs = tokenizer(headline, return_tensors="pt", truncation=True, max_length=128)
outputs = model.generate(**inputs, max_length=128, num_beams=4)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation

Evaluated on a 2,857-sample held-out test split alongside 13 other model variants (BART, T5 baselines, LLaMA 3.2, ablation studies). Metrics include:

Full per-variant numbers are published alongside the Project LLMao webapp.

Related models

• SeeYangZhi/Llama-3.2-1B-Sarcasm-Rewriter — instruction-tuned LLaMA variant
• SeeYangZhi/BART-Base-Sarcasm-Rewriter — supervised baseline
• SeeYangZhi/BART-Base-CE-Sarcasm-Rewriter — context-enhanced SFT
• SeeYangZhi/BART-Base-RL-Sarcasm-Rewriter — REINFORCE on top of baseline
• SeeYangZhi/BART-Base-CE-RL-Sarcasm-Rewriter — CE + RL (best)

License

MIT, inheriting from facebook/bart-base. The NHDSD dataset is used under its original research-use terms.