Downloads · 30 days
11
2% of all-time downloads
mistral-nemo-gutenberg-12B-v4-exl2
CameronRedmore/mistral-nemo-gutenberg-12B-v4-exl2
mistral-nemo-gutenberg-12B-v4-exl2 is a text generation model from CameronRedmore. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains various EXL2 quantisations of nbeerbower/mistral-nemo-gutenberg-12B-v4.
All-time downloads
683
Public
Repo size
60 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.json11.7 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Aug 23, 2024
- Updated
- Aug 23, 2024
- SHA
- 428bfa99
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
mistral-nemo-gutenberg-12B-v4-exl2
This repository contains various EXL2 quantisations of nbeerbower/mistral-nemo-gutenberg-12B-v4.
Quantisations available:
| Branch | Description | Recommended |
|---|---|---|
| 2.0-bpw | 2 bits per weight | Low Quality - Smallest Available Quantisation |
| 3.0-bpw | 3 bits per weight | |
| 4.0-bpw | 4 bits per weight | ✔️ - Recommended for Low-VRAM Environments |
| 5.0-bpw | 5 bits per weight | |
| 6.0-bpw | 6 bits per weight | ✔️ - Best Quality / VRAM Balance |
| 6.5-bpw | 6.5 bits per weight | ✔️ - Near Perfect Quality, Slightly Higher VRAM Usage |
| 8.0-bpw | 8.0 bits per weight | Best Available Quality - Almost always unnecessary |
ORIGINAL README:
TheDrummer/Rocinante-12B-v1 finetuned on jondurbin/gutenberg-dpo-v0.1.
Method
Finetuned using an A100 on Google Colab for 3 epochs.
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Aug 23, 2024
- Updated
- Aug 23, 2024