mt5-Turkish-English-Summarizer

Model Card for Model ID

This is an abstractive text summarization model developed to generate concise, platform-specific content from long-form Turkish SEO articles. The project utilizes the

google/mt5-small architecture, which was efficiently fine-tuned using the LoRA (Low-Rank Adaptation) technique.

The training procedure was executed on a Colab T4 GPU. It involved training for 3 epochs on a substantial Turkish dataset composed of

30,000 training samples and 3,000 validation samples, derived from the XLSum and MLSUM Turkish subsets.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Doğan Bilir
• Funded by [optional]:
• Shared by [optional]: Doğan Bilir
• Model type:
• Language(s) (NLP): Turkish/English
• License:
• Finetuned from model mt-5 small: google/mt5-small

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/jsdogan/document-summarizer-rag
• Demo [optional]: https://huggingface.co/spaces/Doganbilir/turkish-summarizer_mt5?logs=container)

Uses

Generating concise summaries for long Turkish texts, particularly those created by AI content platforms (like Serpsonic), to be used as meta descriptions, TL;DRs, or social media snippets.

Downstream Use [optional]

Can be integrated into a larger NLP pipeline for document analysis and knowledge extraction.

Out-of-Scope Use

Fact-checking, generating content that requires legal or medical expertise.

Bias, Risks, and Limitations

The model inherits the general biases of the multilingual mT5-small model. The summaries might occasionally drop important keywords or struggle with highly technical, domain-specific terminology outside of the training set.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

Training Details

Training Data

A combined dataset consisting of the Turkish subsets of XLSum and MLSUM-TR. The final dataset used for LoRA fine-tuning contained approximately 30,000 Turkish and English samples.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Preprocessing [optional]

Data cleaning included filtering extremely short summaries and overly long documents. Text normalization was applied to fix encoding errors (e.g., “ü” to “ü”). Input length was truncated to 512 tokens.

Training Hyperparameters

• Training regime: Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation) Rank (r)=16, Alpha (α)=32, Dropout=0.05
• Optimization: Learning Rate (LR)=2e-4, Effective Batch Size=16 (via Gradient Accumulation)

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

Metrics

ROUGE-1, ROUGE-2, ROUGE-L (Primary metrics for text summarization).

Results

Summary

Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->