Downloads · 30 days
12
23% of all-time downloads
bogdancazan/t5-small-newsela-biendata-with-domain-adaptation
t5-small-newsela-biendata-with-domain-adaptation is a machine learning model from bogdancazan. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
trainingargs = TrainingArguments( outputdir='t5-small-newsela-biendata-with-domain-adaptation', numtrainepochs=20, warmupsteps=250, perdevicetrainbatchsize=BATCHSIZE, weightdecay=0.01, learningrate=2e-4, fp16=True, op…
Downloads · 30 days
12
23% of all-time downloads
All-time downloads
52
Public
Repo size
727 MB
Likes
0
Public
Click a slice to open those files.
.bin242 MB · 99%
From the Hugging Face model README
training_args = TrainingArguments( output_dir='t5-small-newsela-biendata-with-domain-adaptation', num_train_epochs=20, warmup_steps=250, per_device_train_batch_size=BATCH_SIZE, weight_decay=0.01, learning_rate=2e-4, fp16=True, optim="adafactor", )
Step Training Loss 500 35.466600 1000 25.795400 1500 10.923200 2000 4.515500 TrainOutput(global_step=2320, training_loss=16.92537920721646, metrics={'train_runtime': 628.0033, 'train_samples_per_second': 472.418, 'train_steps_per_second': 3.694, 'total_flos': 0.0, 'train_loss': 16.92537920721646, 'epoch': 20.0})