Downloads · 30 days
7
4% of all-time downloads
Lyra4-Gutenberg-12B_exl2_8bpw_max
DeusImperator/Lyra4-Gutenberg-12B_exl2_8bpw_max
Lyra4-Gutenberg-12B_exl2_8bpw_max is a text generation model from DeusImperator. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a 8bpw EXL2 quant of nbeerbower/Lyra4-Gutenberg-12B
All-time downloads
181
Public
Repo size
13 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors13 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Sep 12, 2024
- Updated
- Sep 12, 2024
- SHA
- 05dd6428
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Lyra4-Gutenberg-12B - EXL2 8bpw max
This is a 8bpw EXL2 quant of nbeerbower/Lyra4-Gutenberg-12B
This quant was made using exllamav2-0.2.1 with default dataset. I used a slightly modified quantization script to force use of highest bpw methods for all layers in the model (which is usually "1:8b_128g s4") to ensure max quality.
I also added a small fix in config file to set max default context at 128k as original Mistral-Nemo should have.
I tested this quant shortly in some random RPs (including ones over 8k context) and it seems to work fine.
Prompt Templates
Uses ChatML or modified mistral format like mentioned in original Lyra v4. I tested it with ChatML.
Original readme below
Lyra4-Gutenberg-12B
Sao10K/MN-12B-Lyra-v4 finetuned on jondurbin/gutenberg-dpo-v0.1.
Method
ORPO Finetuned using an RTX 3090 + 4060 Ti for 3 epochs.
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Sep 12, 2024
- Updated
- Sep 12, 2024