Downloads · 30 days
41
0% of all-time downloads
kaori-34b-v3
KaeriJenti/kaori-34b-v3
kaori-34b-v3 is a text generation model from KaeriJenti. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
All-time downloads
37K
Public
Parameters
34.4B
68.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors68.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Dec 22, 2023
- Updated
- Dec 22, 2023
- SHA
- 117dab7f
Try a prompt
Model card
From the Hugging Face model README
<h1>kaori-34b-v3 Model Card</h1>
This Model was Finetuned By Kaeri and Jenti.
<h3>Datasets Strategy</h3>- Open-Platypus
- Dolphin
We trained the model with 100% Open-Platypus data and 5% Dolphin data and applied SFT strategy.
We did not use GSM8k samples when generating data. Also we were careful of data contamination by similarity filtering the training data if the data correspond to any of the following list.
<pre> filtering_tasks = [ 'cot_gsm8k', 'cot_gsm8k_ii', 'drop:2.0.0', 'winogrande:1.1.0' 'task228_arc_answer_generation_easy', 'ai2_arc/ARC-Challenge:1.0.0', 'ai2_arc/ARC-Easy:1.0.0', 'task229_arc_answer_generation_hard', 'hellaswag:1.1.0', 'task1389_hellaswag_completion' ] </pre> <h3>Framework:</h3> <h3>Parameters:</h3>- Finetune_Type : LoRA
- GPUs : A100x4(80GB)
- Epochs : 3
- Batchsize : 8
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Dec 22, 2023
- Updated
- Dec 22, 2023