flan-t5-large-xsum-peft

Model Info

This is a PEFT version of google/flan-t5-large trained using LoRA with the EdinburghNLP/xsum dataset. The model was trained using 4 A100 GPUs via Accelerate on an HPC cluster.

Hyperparameters

• NUM_EPOCHS = 8
• LEARNING_RATE = 2e-6
• BATCH_SIZE = 8
• OPTIMIZER = AdamW w/ betas=(0.9,0.999), eps=1e-8, weight_decay=0.01
• SCHEDULER = Cosine
• GRAD_ACC_STEPS = 8
• WARMUP_STEPS = 100
• RANK = 16
• LORA_ALPHA = 16
• LORA_DROPOUT = 0.1

Metrics

• Evaluation Loss: 1.6028
• Perplexity: 4.9668
• BERTScore: {'precision': 0.9127, 'recall': 0.9000, 'f1': 0.9062}
• ROUGE Scores: {'rouge1': 0.3933, 'rouge2': 0.1604, 'rougeL': 0.3139, 'rougeLsum': 0.3139}
• BLEU Scores: 0.1002
• Average Generation Length: 26

Example Inference

Prompt:<br/> Jordan Hill, Brittany Covington and Tesfaye Cooper, all 18, and Tanishia Covington, 24, appeared in a Chicago court on Friday. The four have been charged with hate crimes and aggravated kidnapping and battery, among other things. An online fundraiser for their victim has collected $51,000 (£42,500) so far. Denying the four suspects bail, Judge Maria Kuriakos Ciesil asked: "Where was your sense of decency?" Prosecutors told the court the beating started in a van and continued at a house, where the suspects allegedly forced the 18-year-old white victim, who suffers from schizophrenia and attention deficit disorder, to drink toilet water and kiss the floor. Police allege the van was earlier stolen by Mr Hill, who is also accused of demanding $300 from the victim's mother while they held him captive, according to the Chicago Tribune. The court was also told the suspects stuffed a sock into his mouth, taped his mouth shut and bound his hands with a belt. In a video made for Facebook Live which was watched millions of times, the assailants can be heard making derogatory statements against white people and Donald Trump. The victim had been dropped off at a McDonalds to meet Mr Hill - who was one of his friends - on 31 December. He was found by a police officer on Tuesday, 3 January, a day after he was reported missing by his parents. Prosecutors say the suspects each face two hate crimes counts, one because of the victim's race and the other because of his disabilities.

Generated output:<br/> Jordan Hill, Brittany Covington, Tesfaye Cooper and Tanishia Covington are accused of abducting a white man from a McDonalds on 31 December.