mt5-base-thaisum-text-summarization

StelleX/mt5-base-thaisum-text-summarization

mt5-base-thaisum-text-summarization is a summarization model from StelleX. Use it when you need a shorter version of a longer text. It is set up for transformers.

This repository contains the finetuned mT5-base model for Thai sentence summarization. The architecture of the model is based on mT5 model and fine-tuned on text-summarization pairs in Thai.

Downloads · 30 days

1% of all-time downloads

All-time downloads

14.4K

Public

Parameters

582M

7.3 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors2.3 GB · 99%

At a glance

Task: Summarization
Library: transformers
Model type: mt5
Access: Public
Created: Dec 22, 2023
Updated: Sep 16, 2024
SHA: 2488085d

Try a prompt

Model card

From the Hugging Face model README

mt5-base-thaisum

This repository contains the finetuned mT5-base model for Thai sentence summarization. The architecture of the model is based on mT5 model and fine-tuned on text-summarization pairs in Thai.

Example

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

tokenizer = AutoTokenizer.from_pretrained("preechanon/mt5-base-thaisum-text-summarization")
model = AutoModelForSeq2SeqLM.from_pretrained("preechanon/mt5-base-thaisum-text-summarization")
new_input_string = "ข้อความที่ต้องการ"
input_ = tokenizer(new_input_string, truncation=True, max_length=1024, return_tensors="pt")
with torch.no_grad():
    preds = model.generate(
        input_['input_ids'].to('cpu'),
        num_beams=15,
        num_return_sequences=1,
        no_repeat_ngram_size=1,
        remove_invalid_values=True,
        max_length=140,
    )

summary = tokenizer.decode(preds[0], skip_special_tokens=True)
summary

Score

Rouge1: 0.488931
Rouge2: 0.309732
Rougel: 0.425490
Rougelsum: 0.444359

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-04
train_batch_size: 8
eval_batch_size: 1
seed: 42
optimizer: AdamW with betas=(0.9,0.999), epsilon=1e-08 and weight_decay=0.1
warmup step: 5000
lr_scheduler_type: linear
num_epochs: 6
gradient_accumulation_steps: 4

Framework versions

Transformers 4.36.1
Pytorch 2.1.2

Resource Funding

NSTDA Supercomputer center (ThaiSC) and the National e-Science Infrastructure Consortium for their support of computer facilities.

Citation

ปรีชานนท์ ชาติไทย และ สัจจวัจน์ ส่งเสริม. (2567),
การสรุปข้อความข่าวภาษาไทยด้วยโครงข่ายประสาทเทียม (Thai News Text Summarization Using Neural Network),
วิทยาศาสตรบัณฑิต (วทบ.):ขอนแก่น, มหาวิทยาลัยขอนแก่น

Task: Summarization
Library: transformers
Type: mt5
Languages: th
Created: Dec 22, 2023
Updated: Sep 16, 2024