flan-t5-dialogsum-prefix

Flan-T5-Large + prefix-tuning on DialogSum

google/flan-t5-large adapted for dialogue summarization on DialogSum, trained as part of an IIT-D Gen-AI course project comparing four fine-tuning methods under identical conditions.

Method: Prefix-tuning (100 virtual tokens, no prefix projection)

Code, evaluation harness and the other three models: https://github.com/dipika-s/iitd-genai

Known issue

This adapter scores identically to the untuned base model on all 13 evaluation metrics, to four decimal places. Training loss did decrease (1.117 train / 0.948 eval), so the prefix learned something — but it had no effect on generated text, which points at the generation path failing to apply the prefix rather than at a property of prefix-tuning.

The adapter is published as-is for completeness and reproducibility. Do not cite the numbers below as evidence about prefix-tuning without first re-checking generation.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-large")
model = PeftModel.from_pretrained(base, "daggar/flan-t5-dialogsum-prefix")
tokenizer = AutoTokenizer.from_pretrained("daggar/flan-t5-dialogsum-prefix")

dialogue = "#Person1#: Hi, how was your weekend?\n#Person2#: Great, I went hiking."
inputs = tokenizer("Summarize the following dialogue:\n" + dialogue,
                   return_tensors="pt", max_length=512, truncation=True)
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=128)[0],
                       skip_special_tokens=True))

Inputs must use the training prompt — "Summarize the following dialogue:\n" followed by the dialogue — truncated to 512 tokens. Summaries were trained at up to 128 tokens.

Training

Evaluation

Measured on the full 1,500-example DialogSum test split, against the untuned base model.

All four methods

* see the known issue on the prefix model card.

Limitations

Trained only on DialogSum, which is two-speaker English conversation transcripts using #Person1# / #Person2# speaker tags. Summaries of longer, multi-party, domain-specific or non-English dialogue will be unreliable. Dialogues over 512 tokens are truncated, so content late in a long conversation may be dropped. The model inherits any biases present in google/flan-t5-large and in DialogSum, and summaries can contain details not supported by the source dialogue.