Downloads · 30 days
17
0% of all-time downloads
T3Q-Platypus-SOLAR
chlee10/T3Q-Platypus-SOLAR
T3Q-Platypus-SOLAR is a text generation model from chlee10. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
8.2K
Public
Repo size
85.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin42.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Mar 7, 2024
- Updated
- Mar 12, 2024
- SHA
- c86f8fbe
Try a prompt
Base models
Model card
From the Hugging Face model README
Update @ 2024.03.07
T3Q-platypus-SOLAR
This model is a fine-tuned version of upstage/SOLAR-10.7B-v1.0
Model Developers Chihoon Lee(chlee10), T3Q
Training hyperparameters
The following hyperparameters were used during training:
# 데이터셋과 훈련 횟수와 관련된 하이퍼 파라미터
batch_size = 16
num_epochs = 1
micro_batch = 1
gradient_accumulation_steps = batch_size // micro_batch
# 훈련 방법에 대한 하이퍼 파라미터
cutoff_len = 4096
lr_scheduler = 'cosine'
warmup_ratio = 0.06 # warmup_steps = 100
learning_rate = 4e-4
optimizer = 'adamw_torch'
weight_decay = 0.01
max_grad_norm = 1.0
# LoRA config(QLoRA)
lora_r = 16
lora_alpha = 16
lora_dropout = 0.05
lora_target_modules = ["gate_proj", "down_proj", "up_proj"]
# Tokenizer에서 나오는 input값 설정 옵션
train_on_inputs = False
add_eos_token = False
# NEFTune params
noise_alpha: int = 5
Framework versions
- Transformers 4.34.1
- Pytorch 2.1.0+cu121
- Datasets 2.13.0
- Tokenizers 0.14.1
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Mar 7, 2024
- Updated
- Mar 12, 2024