ERNIE-PUYI-FT

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

train_2025-12-23-08-34-28

This model is a fine-tuned version of baidu/ERNIE-4.5-0.3B-PT on the puyi_data dataset.

Model description

This is a fine-tuned version of ERNIE-4.5-0.3B-PT specialized in generating historically accurate responses about Pu Yi (溥儀), the last Emperor of China. The model has been trained on comprehensive historical data covering Pu Yi's life from 1906 to 1967, including his reign during the Qing Dynasty, the Manchukuo period, and his later life in the People's Republic of China.

Intended uses & limitations

Primary Use Cases

• Educational tool for learning about early 20th-century Chinese history
• Interactive Q&A system for historical inquiries about Pu Yi
• Research assistance for historians and students
• Cultural heritage preservation and digital museum applications

Example Queries

• "When did Pu Yi become emperor?"
• "What was Pu Yi's role in Manchukuo?"

Training and evaluation data

The model was fine-tuned on a curated dataset including:

• Wikipedia biographical article
• Historical documents and records
• Academic sources and scholarly articles
• Information about the Qing Dynasty, Manchukuo, and modern Chinese history

Training procedure

• Base Model: ERNIE-4.5-0.3B-PT
• Fine-tuning Method: LoRA
• Epochs: 3.0
• Batch Size: 2
• Learning rate: 0.0003

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 3.0

Training results

Framework versions

• PEFT 0.17.1
• Transformers 4.57.1
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.22.1