Downloads · 30 days
10
10% of all-time downloads
adamo1139/Yi-34B-200K-XLCTX-RAW-ORPO-0805-GaLore
Yi-34B-200K-XLCTX-RAW-ORPO-0805-GaLore is a text generation model from adamo1139. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Finetune of Yi-34B-200K (the version with better ctx, Yi-34B-200K v2 or Yi-34B-200K-XLCTX (my preffered name)) on adamo1139/rawrrv22stage1 dataset via ORPO and GaLore on 4-bit (bnb) weights.
Downloads · 30 days
10
10% of all-time downloads
All-time downloads
96
Public
Parameters
34.4B
68.8 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors68.8 GB · 100%
From the Hugging Face model README
Finetune of Yi-34B-200K (the version with better ctx, Yi-34B-200K v2 or Yi-34B-200K-XLCTX (my preffered name)) on adamo1139/rawrr_v2_2_stage1 dataset via ORPO and GaLore on 4-bit (bnb) weights.
This is not a chat model!! It's meant to serve as base for further finetuning that has less behaviour inherited from being trained on OpenAI etc. AI generated content. If you don't want your finetune to sound like an AI model, using this as a base should be a good idea.