Qwen2.5-3B-Instruct-SLDS

Model Card for Qwen2.5-3B-Instruct-SLDS

Model Summary

This model is a Qwen2.5-3B-Instruct fine-tuned on the Swiss Landmark Decisions Summarization (SLDS) dataset.
SLDS is a multilingual dataset of 20,000 Swiss Federal Supreme Court decisions (1954–2024), each paired with headnotes in German, French, and Italian, resulting in ~60,000 decision–headnote pairs.

The model is optimized for legal abstractive summarization and is capable of producing concise, legally structured headnotes.
It can be used for both monolingual and cross-lingual summarization tasks.

This model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

---

Intended Use

• Primary Task: Judicial summarization (decision → headnote generation).
• Languages: German (de), French (fr), Italian (it).
• Scenarios:
  • Monolingual summarization: e.g., German decision → German headnote.
  • Cross-lingual summarization: e.g., German decision → French headnote.
  • Legal research support: assisting in retrieval and navigation of court decisions.

Not intended for:

• Replacing human legal expertise.
• Serving as an authoritative legal source.
• Automated legal advice or decision-making.

---

Training Data

• Dataset: Swiss Landmark Decisions Summarization (SLDS).
• Size: ~20K decisions, ~60K decision–headnote pairs.
• Splits: Train (1954–2021), Validation (2022), Test (2023–2024).
• Source: Swiss Federal Supreme Court.

---

Training Procedure

• Base Models:

  • Qwen2.5 family (0.5B–14B)
  • Llama 3.2 (3B)
  • Phi-3.5-mini

• Fine-tuning Objective: Conditional generation (decision → headnote).

• Evaluation Metrics:

  • Lexical: ROUGE-1/2/L, BLEU, BERTScore.
  • Domain-specific: LLM-as-a-Judge framework (DeepSeek V3) assessing five rubrics: accuracy, completeness, clarity, legal citations, and considerations.

---

Model Performance

On the SLDS test set (2023–2024):

• Lexical metrics: Fine-tuned models outperform in overlap-based scores.
• LLM-judge scores: Larger proprietary and reasoning models outperform in legal precision.

---

Limitations

• Language imbalance: German decisions dominate, while Italian remains underrepresented.
• Biases: Headnotes reflect judicial style and conventions, not neutral summaries.
• Evaluation mismatch: ROUGE and BLEU may not fully capture legal accuracy.
• Overfitting risk: Models may overfit to formulaic headnote structures.
• Cross-lingual difficulty: Some models struggle with non-monolingual headnote generation.

---

Ethical Considerations

• Sensitive information: All data is anonymized by the Swiss Federal Supreme Court before publication.
• Legal risk: Generated headnotes must not be used as official legal advice.
• Fair use: Ensure attribution when reusing outputs.

---

How to Cite

If you use this model, please cite the dataset paper:

@inproceedings{rolshoven-etal-2025-unlocking,
    title = "Unlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in {S}witzerland",
    author = {Rolshoven, Luca  and
      Rasiah, Vishvaksenan  and
      Bose, Srinanda Br{\"u}gger  and
      Hostettler, Sarah  and
      Burkhalter, Lara  and
      St{\"u}rmer, Matthias  and
      Niklaus, Joel},
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.findings-emnlp.832/",
    pages = "15382--15411",
    ISBN = "979-8-89176-335-7",
}