Downloads · 30 days
7
30% of all-time downloads
CorticalStack_-_neurotic-crown-clown-7b-tak-stack-dpo-4bits
RichardErkhov/CorticalStack_-_neurotic-crown-clown-7b-tak-stack-dpo-4bits
CorticalStack_-_neurotic-crown-clown-7b-tak-stack-dpo-4bits is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
23
Public
Parameters
7.4B
4.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.5 GB · 100%
Parameter types
How the weights are stored.
U87B · 95%
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
neurotic-crown-clown-7b-tak-stack-dpo - bnb 4bits
- Model creator: https://huggingface.co/CorticalStack/
- Original model: https://huggingface.co/CorticalStack/neurotic-crown-clown-7b-tak-stack-dpo/
Original model description:
license: apache-2.0 tags:
- dpo base_model:
- CorticalStack/neurotic-crown-clown-7b-ties dataset:
- CorticalStack/tak-stack-dpo
<img src="neurotic-crown-clown-tak-stack.png" alt="Neurotic crown clown tak stack logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
neurotic-crown-clown-7b-tak-stack-dpo
neurotic-crown-clown-7b-tak-stack-dpo is a DPO fine-tuned version of CorticalStack/neurotic-crown-clown-7b-ties using the CorticalStack/tak-stack-dpo dataset.
LoRA
- r: 32
- LoRA alpha: 32
- LoRA dropout: 0.05
Training arguments
- Batch size: 4
- Gradient accumulation steps: 4
- Optimizer: paged_adamw_32bit
- Max steps: 100
- Learning rate: 5e-05
- Learning rate scheduler type: cosine
- Beta: 0.1
- Max prompt length: 1024
- Max length: 1536
- Type
- mistral
- Created
- Sep 14, 2024
- Updated
- Sep 14, 2024