Downloads · 30 days
4
2% of all-time downloads
karakuri-lm-70b-v0.1-exl2
MatrixC7/karakuri-lm-70b-v0.1-exl2
karakuri-lm-70b-v0.1-exl2 is a text generation model from MatrixC7. Use it when you need the model to write or continue text. The card lists the license as cc-by-sa-4.0.
This repository contains the exl2 quants of karakuri-ai/karakuri-lm-70b-v0.1, calibrated using the default dataset built by exllamav2.
All-time downloads
212
Public
Repo size
338 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json4.5 MB · 100%
At a glance
- Task
- Text Generation
- License
- cc-by-sa-4.0
- Access
- Public
- Created
- Feb 2, 2024
- Updated
- Feb 3, 2024
- SHA
- 2ccc3b32
Base models
Model card
From the Hugging Face model README
This repository contains the exl2 quants of karakuri-ai/karakuri-lm-70b-v0.1, calibrated using the default dataset built by exllamav2.
Compatible with exllamav2 version 0.0.11 and later. For optimal model loading, it is recommended to use tabbyAPI.
The measurement file is attached in the branch main and all quants are stored in their respective branches.
The chart below presents the calibration perplexity and wikitext-103-v1 test perplexity for all provided quants.
| Quant | Calibration Perplexity | wikitext-103-v1 Test Perplexity |
|---|---|---|
| 2.4bpw-h6 | 8.4726 | 7.1337 |
| 2.65bpw-h6 | 8.0901 | 6.5724 |
| 3.0bpw-h6 | 7.9927 | 6.4607 |
| 4.0bpw-h6 | 7.6440 | 5.8014 |
| 4.65bpw-h6 | 7.5872 | 5.7112 |
| 5.0bpw-h6 | 7.5745 | |
| 6.0bpw-h6 | 7.5616 | |
| 8.0bpw-h8 | 7.5604 |
*: This first 2.65bpw-h6 quantized version has been deprecated due to unexpectedly high wikitext-103-v1 Test Perplexity values.
- Task
- Text Generation
- License
- cc-by-sa-4.0
- Languages
- ja, en
- Created
- Feb 2, 2024
- Updated
- Feb 3, 2024