Downloads · 30 days
0
Trendyol-LLM-7b-chat-v1.0-RLHF
umarigan/Trendyol-LLM-7b-chat-v1.0-RLHF
Trendyol-LLM-7b-chat-v1.0-RLHF is a question answering model from umarigan. Use it when the input is a question plus a passage. It is set up for adapter-transformers. The card lists the license as apache-2.0.
- Developed by: umarigan - License: apache-2.0 - Finetuned from model : Trendyol/Trendyol-LLM-7b-chat-v1.0
Access
Public
Updated Mar 16, 2024
Parameters
7.3B
14.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.7 GB · 100%
At a glance
- Task
- Question Answering
- Library
- adapter-transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Mar 16, 2024
- Updated
- Mar 16, 2024
- SHA
- 3da4709f
Try a prompt
Base models
Model card
From the Hugging Face model README
Uploaded model
- Developed by: umarigan
- License: apache-2.0
- Finetuned from model : Trendyol/Trendyol-LLM-7b-chat-v1.0
This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.
I used 10k pair of Turkish rlhf dataset.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1
Training results
Framework versions
- PEFT 0.9.0
- Transformers 4.38.2
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Task
- Question Answering
- Library
- adapter-transformers
- Type
- mistral
- License
- apache-2.0
- Languages
- en, tr
- Created
- Mar 16, 2024
- Updated
- Mar 16, 2024