mbart-large-50-extractive-conclusion

mbart-large-50-extractive-conclusion

A fine-tuned mBART-large-50 model for extractive conclusion generation.

Model Description

This model is fine-tuned from facebook/mbart-large-50 for the task of extracting conclusions from long-form text answers. The model learns to identify and extract the key conclusion sentences that are already present in the input text.

Task

Given a long answer text containing reasoning steps and explanations, the model extracts the final conclusion sentence(s) that summarize the answer.

Training Details

• Base Model: facebook/mbart-large-50
• Task: Extractive Conclusion Generation
• Training Framework: HuggingFace Transformers + Seq2SeqTrainer

Hyperparameters

• Learning Rate: 1e-5
• Batch Size: 4 (per device)
• Gradient Accumulation Steps: 16
• Optimizer: AdamW
• LR Scheduler: Linear
• Precision: BF16

Performance

Evaluated on 100 test samples:

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "XiaHan19/mbart-large-50-extractive-conclusion"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# Prepare input
text = "Your long answer text here..."
inputs = tokenizer(text, return_tensors="pt", max_length=1024, truncation=True)

# Generate conclusion
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_length=256,
        num_beams=4,
        early_stopping=False
    )

conclusion = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(conclusion)

Limitations

• The model works best when the input text contains a clear conclusion
• Maximum input length is 1024 tokens
• Maximum output length is 256 tokens

License

MIT License