Downloads · 30 days
65
0% of all-time downloads
gemma-7b-ultrachat-sft
CorticalStack/gemma-7b-ultrachat-sft
gemma-7b-ultrachat-sft is a text generation model from CorticalStack. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
gemma-7b-ultrachat-sft is an SFT fine-tuned version of google/gemma-7b using the stingning/ultrachat dataset.
All-time downloads
14.6K
Public
Parameters
8.5B
17.1 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors17.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- gemma
- Access
- Public
- Created
- Feb 22, 2024
- Updated
- Feb 22, 2024
- SHA
- c4cd06ea
Try a prompt
Model card
From the Hugging Face model README
gemma-7b-ultrachat-sft
gemma-7b-ultrachat-sft is an SFT fine-tuned version of google/gemma-7b using the stingning/ultrachat dataset.
Fine-tuning configuration
LoRA
- LoRA r: 8
- LoRA alpha: 16
- LoRA dropout: 0.1
Training arguments
- Epochs: 1
- Batch size: 4
- Gradient accumulation steps: 6
- Optimizer: paged_adamw_32bit
- Max steps: 100
- Learning rate: 0.0002
- Weight decay: 0.001
- Learning rate scheduler type: constant
- Max seq length: 2048
- Task
- Text Generation
- Library
- transformers
- Type
- gemma
- License
- apache-2.0
- Created
- Feb 22, 2024
- Updated
- Feb 22, 2024