Downloads · 30 days
0
yancongwen/chatglm2-6b-pt-16-2e-2-20230713
chatglm2-6b-pt-16-2e-2-20230713 is a machine learning model from yancongwen. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
0
Access
Public
Updated Jul 18, 2023
Repo size
3.9 MB
Likes
0
Public
Click a slice to open those files.
.pt1.8 MB · 44%
From the Hugging Face model README
具体参数如下:
PRE_SEQ_LEN=16
LR=2e-2
NUM_GPUS=1
torchrun --standalone --nnodes=1 --nproc-per-node=$NUM_GPUS main.py \
--do_train \
--train_file train_data/train_100k.json \
--validation_file train_data/dev_1k.json \
--preprocessing_num_workers 10 \
--prompt_column question \
--response_column answer \
--overwrite_cache \
--model_name_or_path THUDM/chatglm2-6b \
--output_dir output/adgen-chatglm2-6b-pt-$PRE_SEQ_LEN-$LR \
--overwrite_output_dir \
--max_source_length 256 \
--max_target_length 512 \
--per_device_train_batch_size 2 \
--per_device_eval_batch_size 2 \
--gradient_accumulation_steps 8 \
--predict_with_generate \
--max_steps 1000 \
--logging_steps 10 \
--save_steps 1000 \
--learning_rate $LR \
--pre_seq_len $PRE_SEQ_LEN \
--quantization_bit 4
***** train metrics *****
epoch = 0.2
train_loss = 0.1362
train_runtime = 3:10:34.05
train_samples = 78577
train_samples_per_second = 1.399
train_steps_per_second = 0.087
license: unlicense
language:
tags: