T5-base-adapter-Xsum

T5-base with AdaLoRA Fine-Tuned on XSum

Model Description

This model is a fine-tuned version of flax-community/t5-base-cnn-dm on the XSum dataset using AdaLoRA (Adaptive Low-Rank Adaptation) for parameter-efficient fine-tuning.
It was trained with Hugging Face Seq2SeqTrainer for 15 epochs.

• Base model: flax-community/t5-base-cnn-dm
• Fine-tuning method: AdaLoRA (PEFT)
• Task: Abstractive summarization
• Dataset: XSum (Extreme Summarization)
• Languages: English

---

How to Use

You can load and use the full merged model for inference:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "mia-project-2025/T5-base-adapter-Xsum"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

text = """The prime minister gave a long speech in parliament today, discussing the
future of the economy and outlining new strategies to boost growth."""

inputs = tokenizer("summarize: " + text, return_tensors="pt", max_length=512, truncation=True)

summary_ids = model.generate(inputs["input_ids"], attention_mask=inputs["attention_mask"], max_length=64)
print(tokenizer.decode(summary_ids[0], skip_special_tokens=True))

---

## Results
Final Evaluation on XSum (Validation)
Metric	Score
-ROUGE-1	|28.84
-ROUGE-2	|8.01
-ROUGE-L	|22.65
-ROUGE-Lsum	|22.66
-Generation Length	|19.67
-Evaluation Loss	| 0.2899