Downloads · 30 days
0
yancongwen/chatglm2-6b-pt-16-1e-2-20230721-2
chatglm2-6b-pt-16-1e-2-20230721-2 is a machine learning model from yancongwen. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
0
Access
Public
Updated Jul 21, 2023
Repo size
3.8 MB
Likes
0
Public
Click a slice to open those files.
.pt1.8 MB · 47%
From the Hugging Face model README
PRE_SEQ_LEN=16
LR=1e-2
NUM_GPUS=1
torchrun --standalone --nnodes=1 --nproc-per-node=$NUM_GPUS main.py \
--do_train \
--train_file train_data/train_100k.json \
--validation_file train_data/dev_1k.json \
--preprocessing_num_workers 10 \
--prompt_column question \
--response_column answer \
--overwrite_cache \
--model_name_or_path THUDM/chatglm2-6b \
--output_dir output/chatglm2-6b-pt-$PRE_SEQ_LEN-$LR-20230721-2 \
--overwrite_output_dir \
--max_source_length 256 \
--max_target_length 128 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--predict_with_generate \
--max_steps 1000 \
--logging_steps 10 \
--save_steps 1000 \
--learning_rate $LR \
--pre_seq_len $PRE_SEQ_LEN \
--quantization_bit 4
{
"epoch": 0.1,
"train_loss": 0.18890315681695938,
"train_runtime": 3951.0431,
"train_samples": 78577,
"train_samples_per_second": 2.025,
"train_steps_per_second": 0.253
}