Downloads · 30 days
0
gptj6b-lora-owca
kil3r/gptj6b-lora-owca
gptj6b-lora-owca is a text generation model from kil3r. Use it when you need the model to write or continue text. The card lists the license as unlicense.
Access
Public
Updated Apr 24, 2023
Repo size
7.4 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin7.4 MB · 100%
At a glance
- Task
- Text Generation
- License
- unlicense
- Access
- Public
- Created
- Apr 23, 2023
- Updated
- Apr 24, 2023
- SHA
- ef2c9987
Datasets
Model card
From the Hugging Face model README
This repo contains EleutherAI/gpt-j-6B fine tuned on OWCA (https://github.com/Emplocity/owca) using LoRa
Training params:<br/>
MICRO_BATCH_SIZE = 64
BATCH_SIZE = 128
GRADIENT_ACCUMULATION_STEPS = BATCH_SIZE // MICRO_BATCH_SIZE
EPOCHS = 3
LEARNING_RATE = 2e-5
CUTOFF_LEN = 256
LORA_R = 4
LORA_ALPHA = 16
LORA_DROPOUT = 0.05
warmup_steps=100
fp16=True
- Task
- Text Generation
- License
- unlicense
- Languages
- pl
- Created
- Apr 23, 2023
- Updated
- Apr 24, 2023