Downloads · 30 days
0
airoboros-2.2.1-limarpv3-y34b-exl2
DS-Archive/airoboros-2.2.1-limarpv3-y34b-exl2
airoboros-2.2.1-limarpv3-y34b-exl2 is a text generation model from DS-Archive. Use it when you need the model to write or continue text. The card lists the license as other.
Exllama v2 quant of Doctor-Shotgun/airoboros-2.2.1-limarpv3-y34b
Access
Public
Updated Nov 11, 2023
Repo size
47.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json6.6 MB · 100%
At a glance
- Task
- Text Generation
- License
- other
- Access
- Public
- Created
- Nov 11, 2023
- Updated
- Nov 11, 2023
- SHA
- a3685275
Try a prompt
Model card
From the Hugging Face model README
airoboros-2.2.1-limarpv3-y34b-exl2
Exllama v2 quant of Doctor-Shotgun/airoboros-2.2.1-limarpv3-y34b
Branches:
- main: measurement.json calculated at 2048 token calibration rows on PIPPA
- 4.65bpw-h6: 4.65 decoder bits per weight, 6 head bits
- ideal for 24gb GPUs at 8k context (on my 24gb Windows setup with flash attention 2, peak VRAM usage during inference with exllamav2_hf was around 23.4gb with 0.9gb used at baseline)
- 6.0bpw-h6: 6 decoder bits per weight, 6 head bits
- ideal for large (>24gb) VRAM setups
- Task
- Text Generation
- License
- other
- Languages
- en
- Created
- Nov 11, 2023
- Updated
- Nov 11, 2023