mbart-arabic-summarizer

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

mbart_summarizer_xsum

This model is a fine-tuned version of facebook/mbart-large-50 on asas-ai/Arabic-article-summarization dataset.

Model description

• Base Model: facebook/mbart-large-50
• Task: Abstractive summarization
• Dataset: Arabic-article-summarization
• Evaluation Metric: ROUGE (1/2/L/Lsum)
• Mixed Precision: Yes (fp16)

Intended uses & limitations

Only Fine Tuned for 3 epochs , and the data is not in very good quality, better to take and fine tune more on better data quality depending on the use case.

Training and evaluation data

Evaluation Metric: ROUGE (1/2/L/Lsum) trained and evaluated on the [asas-ai/Arabic-article-summarization] dataset which i splitted into training and testing splts

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

After 3 Epochs of Training:

• ROUGE1: 41.2430
• ROUGE2: 24.7636
• ROUGEL: 40.6967
• ROUGELSUM: 40.9913

Usage

First Approach

<pre lang="python"><code> ```python from transformers import MBartForConditionalGeneration, MBartTokenizer # Load model and tokenizer from Hugging Face Hub model_name = "karimraouf/mbart-arabic-summarizer" tokenizer = MBartTokenizer.from_pretrained(model_name) model = MBartForConditionalGeneration.from_pretrained(model_name) # Input text (replace with your own) input_text = "أي نص تريده" # Tokenize the input inputs = tokenizer( input_text, return_tensors="pt", max_length=1024, truncation=True, padding=True ) # Generate the summary summary_ids = model.generate( **inputs, max_length=200, num_beams=3, early_stopping=True ) # Decode and print the summary summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True) print("Summary:", summary) </code></pre>

Second Approach

<pre lang="python"><code> ```python from transformers import pipeline # Load the summarization pipeline with the pre-trained Arabic MBart model summarizer = pipeline( "summarization", model="karimraouf/mbart-arabic-summarizer", tokenizer="karimraouf/mbart-arabic-summarizer" ) # Replace this with your own Arabic text input_text = "أي نص تريده" # Generate the summary summary = summarizer( input_text, max_length=200, min_length=30, do_sample=False ) # Print the result print(summary[0]['summary_text']) </code></pre>

Framework versions

• Transformers 4.52.4
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1