Downloads · 30 days
12
13% of all-time downloads
T3Q-Platypus-Mistral7B
chlee10/T3Q-Platypus-Mistral7B
T3Q-Platypus-Mistral7B is a text generation model from chlee10. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
96
Public
Repo size
57.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin29 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Mar 12, 2024
- Updated
- Mar 12, 2024
- SHA
- 266e1517
Try a prompt
Base models
Model card
From the Hugging Face model README
Update @ 2024.03.07
T3Q-Platypus-Mistral7B
This model is a fine-tuned version of bardsai/jaskier-7b-dpo-v6.1
Model Developers Chihoon Lee(chlee10), T3Q
Training hyperparameters
The following hyperparameters were used during training:
# 데이터셋과 훈련 횟수와 관련된 하이퍼 파라미터
batch_size = 16
num_epochs = 1
micro_batch = 1
gradient_accumulation_steps = batch_size // micro_batch
# 훈련 방법에 대한 하이퍼 파라미터
cutoff_len = 4096
lr_scheduler = 'cosine'
warmup_ratio = 0.06 # warmup_steps = 100
learning_rate = 4e-4
optimizer = 'adamw_torch'
weight_decay = 0.01
max_grad_norm = 1.0
# LoRA config
lora_r = 16
lora_alpha = 16
lora_dropout = 0.05
lora_target_modules = ["gate_proj", "down_proj", "up_proj"]
# Tokenizer에서 나오는 input값 설정 옵션
train_on_inputs = False
add_eos_token = False
# NEFTune params
noise_alpha: int = 5
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Languages
- en
- Created
- Mar 12, 2024
- Updated
- Mar 12, 2024