Downloads · 30 days
8
1% of all-time downloads
heni86/magnum-72b-v1-exl2-rpcal
magnum-72b-v1-exl2-rpcal is a text generation model from heni86. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Quantized using the cleaned PIPPA roleplay dataset.
Downloads · 30 days
8
1% of all-time downloads
All-time downloads
725
Public
Repo size
322 GB
Likes
4
Public
Click a slice to open those files.
.json14.3 MB · 89%
From the Hugging Face model README
Quantized using the cleaned PIPPA roleplay dataset.
2.25bpw6h quants (tested and working on a single RTX 3090 24GiB at 16k context length)
2.4bpw6h quants (may not load on 24GiB VRAM machines!)
4.0bpw8h quants (tested and working on two 3090s at 32k context/cache)
4.4bpw8h quants (tested and working on two 3090s at 32k context, 64k Q4 cache (for CFG or parallelism) with tabbyAPI)
All tests performed on a headless Linux instance with no active desktop environment to maximize VRAM.
Other quants available on request, feel free to ask!
See original model for further details.

This is the first in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus. This model is fine-tuned on top of Qwen-2 72B Instruct.
Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:
"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""
This model has been a team effort, credits go to:
And last but not least, we'd like to thank Kearm for sponsoring the compute needed to train this model.
The training was done with 55 million tokens of high-quality RP data, over 1.5 epochs. We used 8x AMD Instinct™ MI300X Accelerators for the full-parameter fine-tuning of the model.
...