Downloads · 30 days
10
5% of all-time downloads
adamo1139/Yi-34B-200K-HESOYAM-rawrr_stage2-2306
Yi-34B-200K-HESOYAM-rawrr_stage2-2306 is a text generation model from adamo1139. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
1 epoch on adamo1139/uninstruct-v1-experimental-chatml, then 1 epoch on adamo1139/HESOYAMv0.3, then a fraction of an epoch on adamo1139/rawrrv2-1-stage2. I used GaLore for all three stages.
Downloads · 30 days
10
5% of all-time downloads
All-time downloads
218
Public
Parameters
34.4B
68.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors68.8 GB · 100%
From the Hugging Face model README
1 epoch on adamo1139/uninstruct-v1-experimental-chatml, then 1 epoch on adamo1139/HESOYAM_v0.3, then a fraction of an epoch on adamo1139/rawrr_v2-1-stage2. I used GaLore for all three stages.
After I saw that adamo1139/Yi-34B-200K-HESOYAM-2206 is not free of slop if you don't use the right prompt, I decided to try to ORPO it out using adamo1139/rawrr_v2-1-stage2.
The effects are mixed - it's pleasant to talk to, it doesn't really feel like you're exchanging comments on reddit/4chan anymore. It's very easy to have a nice human-like discussion with it, it has potential.
Prompt format is ChatML, not sure which system prompt works best.