Downloads · 30 days
0
0% of all-time downloads
kepton0117/chrono-2021-ckpts
chrono-2021-ckpts is a text generation model from kepton0117. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Continue-pretrain of anacoluthe89/chrono-2020 @633eb832 on kepton0117/sn38-train-2021.
Downloads · 30 days
0
0% of all-time downloads
All-time downloads
1
Public
Repo size
4 GB
Likes
0
Public
Click a slice to open those files.
Other1.5 KB · 58%
From the Hugging Face model README
Continue-pretrain of anacoluthe89/chrono-2020 @633eb832 on kepton0117/sn38-train-2021.
On-chain pin is not this repo. UID 141 uses kepton0117/chrono-2021-sn38 (step-4000 only).
Each revision is one optimizer checkpoint (1e-3 stage, init = 1e-4 step-4000):
| Revision | Cosine vs UID 131 | Loss ~ |
|---|---|---|
step-2000 | 0.9538 (fail copy) | 2.8 |
step-4000 | 0.9294 | 2.66 |
step-6000 | 0.9144 | — |
step-8000 | 0.9080 | 2.54–2.61 |
step-10000 | 0.9061 | — |
step-12000 | 0.9055 | — |
step-14000 | 0.9055 | — |
step-15000 | 0.9055 | 2.50 |
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained(
"kepton0117/chrono-2021-ckpts", revision="step-8000", trust_remote_code=False,
)