t5-id-summarization-augmented3x

T5 Indonesian Summarization (Augmented 3x)

Model T5-base yang di-fine-tune dengan augmentasi data 3x (utterance-level paraphrase) untuk meringkas percakapan Bahasa Indonesia. Training menggunakan 5-Fold Cross Validation.

Model Details

• Base Model: cahya/t5-base-indonesian-summarization-cased
• Architecture: T5-base (encoder-decoder, 12 layers, 768 hidden, 12 heads)
• Parameters: ~220M
• Language: Indonesian (Bahasa Indonesia)
• Task: Abstractive Summarization of Indonesian Conversations
• Training: 5-Fold Cross Validation
• Available Folds: 5 folds tersedia sebagai branches (fold_0 s/d fold_4). Branch main berisi fold 3 (performa terbaik).

Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

# Load model & tokenizer
model_name = "aloisiusedwin/t5-id-summarization-augmented3x"
tokenizer = T5Tokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

# Contoh percakapan
conversation = "summarize: S1: Halo, gimana kabarmu? S2: Baik, aku lagi sibuk ngerjain tugas nih."

# Generate ringkasan
inputs = tokenizer(conversation, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(
    inputs["input_ids"],
    max_length=150,
    num_beams=1,
    no_repeat_ngram_size=2,
    early_stopping=True
)
summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(summary)

Loading Specific Fold

# Load fold tertentu (misal fold_0)
model = T5ForConditionalGeneration.from_pretrained(model_name, revision="fold_0")
tokenizer = T5Tokenizer.from_pretrained(model_name, revision="fold_0")

Training Details

Hyperparameters

Data Augmentation

Dataset diperbesar 3x dengan teknik utterance-level paraphrase menggunakan model Wikidepia/IndoT5-base-paraphrase. Setiap percakapan di-paraphrase secara per-kalimat.

Evaluation Results

Per-Fold Results

(best) = Fold terbaik (digunakan sebagai branch main)

Aggregated (5-Fold Cross Validation)

Perbandingan dengan Baseline

Intended Use

Model ini dirancang untuk meringkas percakapan dalam Bahasa Indonesia.

Limitations

• Input harus diawali dengan prefix summarize: untuk hasil optimal.
• Panjang input maksimum 512 token.

Citation

@misc{putra2026finetuning,
  author       = {Putra, A. Edwin Krisandika},
  title        = {Fine-tuning Model Transformer T5 untuk Abstractive Text Summarization pada Percakapan Berbahasa Indonesia},
  year         = {2026},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/aloisiusedwin/TA_AEdwin}},
  note         = {Tugas Akhir, Program Studi Teknik Informatika, Institut Teknologi Sumatera}
}