Downloads · 30 days
5
10% of all-time downloads
benk04/CausalLM-RP-34B-4.65bpw-h6-exl2
CausalLM-RP-34B-4.65bpw-h6-exl2 is a text generation model from benk04. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Exllamav2 4.65bpw quantization of CausalLM-RP-34B from NeverSleep, quantized with default calibration dataset. [!IMPORTANT] Fits in 24GB VRAM with 32k+ context. Make sure to enable 4-bit cache option or you'll run int…
Downloads · 30 days
5
10% of all-time downloads
All-time downloads
48
Public
Repo size
20.7 GB
Likes
0
Public
Click a slice to open those files.
.safetensors20.7 GB · 100%
From the Hugging Face model README
Exllamav2 4.65bpw quantization of CausalLM-RP-34B from NeverSleep, quantized with default calibration dataset.
[!IMPORTANT] Fits in 24GB VRAM with 32k+ context. Make sure to enable 4-bit cache option or you'll run into OOM errors.
This repo contains fp16 files of CausalLM-RP-34B, a finetuned model of the CausalLM-34B Beta on multiple RP datasets.
<!-- description end --> <!-- description start --><|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>