SummOrchestra-Qwen3-8B-GRPO-BRLP-SAMSUM

dadastory/SummOrchestra-Qwen3-8B-GRPO-BRLP-SAMSUM is a dialogue-summarization model trained on the SamSum dataset through a two-stage pipeline combining Supervised Fine-Tuning (SFT) and GRPO + BRLP preference optimization.
This design yields a model with higher faithfulness, reduced hallucination, and stronger alignment with human preference judgments.

Performance Comparison

The model consistently outperforms standard SamSum-style summarizers in both loyalty and human preference metrics.

<p align="center"> <img src="./barplot_comparison.png" alt="Barplot Comparison" width="70%"> </p> <p align="center"> <img src="./kde_faithfulness_comparison.png" alt="Kde Faithfulness Comparison" width="70%"> </p> <p align="center"> <img src="./kde_human_preference_comparison.png" alt="Kde Human Preference Comparison" width="70%"> </p>

Training Overview

1. Supervised Fine-Tuning (SFT)

• Trained on SamSum to establish strong grounding and structural summarization ability.

2. GRPO + BRLP Preference Optimization

• GRPO introduces preference-driven refinement.
• BRLP (Behavioral Reward Learning) provides stable reward shaping.

Evaluation & Alignment

Loyalty / Faithfulness Evaluation

Faithfulness is evaluated using vectara/hallucination_evaluation_model, ensuring:

• Lower hallucination rate
• Higher semantic grounding
• More accurate retention of source dialogue meaning

Human Preference Evaluation

Human-likeness and preference alignment are assessed using Qwen/WorldPM-72B-RLHFLow, focusing on:

• Natural summarization style
• Alignment with human judgment patterns
• Improved preference scoring relative to SamSum baselines

Key Characteristics

• Base Model: Qwen3-8B
• Dataset: SamSum dialogues
• Training Stages: SFT + GRPO + BRLP
• Goals:
  • Enhance faithfulness
  • Improve key information coverage
  • Reduce hallucination
  • Increase human preference alignment