470_FINAL_TECHSIMPLIFY

results

This model is a fine-tuned version of t5-small on the dataset: https://huggingface.co/datasets/sentence-transformers/simple-wiki. It achieves the following results on the evaluation set:

• eval_loss: 1.2573
• eval_model_preparation_time: 0.0027
• eval_rouge1: 54.7902
• eval_rouge2: 40.4054
• eval_rougeL: 51.7529
• eval_rougeLsum: 51.7451
• eval_runtime: 1078.8925
• eval_samples_per_second: 18.95
• eval_steps_per_second: 2.369
• step: 0

Model description

This is a fine tuned T5 based model that was trained upon the wiki-simple dataset: https://huggingface.co/datasets/sentence-transformers/simple-wiki. The idea is to build a self-service technical simplification model for my Machine Learning course final project!

Intended uses & limitations

Intended use cases would be simplifying user-inputted text for one application, or producing bullet point summaries that are also accordingly simplified. The biggest limitation was time constraints and the small size of the dataset. Also to note, it is trained upon wikipedia. This means it may fall short when dealing with medical or legal terminology.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2