Downloads · 30 days
0
yancongwen/chatglm2-6b-pt-16-1e-2-20230720-2
chatglm2-6b-pt-16-1e-2-20230720-2 is a machine learning model from yancongwen. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
0
Access
Public
Updated Jul 20, 2023
Repo size
3.8 MB
Likes
0
Public
Click a slice to open those files.
.pt1.8 MB · 47%
From the Hugging Face model README
PRE_SEQ_LEN=16
LR=1e-2
NUM_GPUS=1
torchrun --standalone --nnodes=1 --nproc-per-node=$NUM_GPUS main.py \
--do_train \
--train_file train_data/train_100k.json \
--validation_file train_data/dev_1k.json \
--preprocessing_num_workers 10 \
--prompt_column question \
--response_column answer \
--overwrite_cache \
--model_name_or_path THUDM/chatglm2-6b \
--output_dir output/chatglm2-6b-pt-$PRE_SEQ_LEN-$LR-20230720-2 \
--overwrite_output_dir \
--max_source_length 256 \
--max_target_length 128 \
--per_device_train_batch_size 2 \
--per_device_eval_batch_size 2 \
--gradient_accumulation_steps 8 \
--predict_with_generate \
--max_steps 1000 \
--logging_steps 10 \
--save_steps 1000 \
--learning_rate $LR \
--pre_seq_len $PRE_SEQ_LEN \
--quantization_bit 4
epoch = 0.2
train_loss = 0.1803
train_runtime = 1:44:48.92
train_samples = 78577
train_samples_per_second = 2.544
train_steps_per_second = 0.159