Downloads · 30 days
8
12% of all-time downloads
DBMe/magnum-v2-123b-2.85bpw-h6-exl2
magnum-v2-123b-2.85bpw-h6-exl2 is a text generation model from DBMe. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Quantized model = https://huggingface.co/anthracite-org/magnum-v2-123b
Downloads · 30 days
8
12% of all-time downloads
All-time downloads
68
Public
Repo size
44.5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors44.5 GB · 100%
From the Hugging Face model README
Quantized model => https://huggingface.co/anthracite-org/magnum-v2-123b
Quantization Details:
Quantization is done using turboderp's ExLlamaV2 v0.2.2.
I use the default calibration datasets and arguments. The repo also includes a "measurement.json" file, which was used during the quantization process.
For models with bits per weight (BPW) over 6.0, I default to quantizing the lm_head layer at 8 bits instead of the standard 6 bits.
Who are you? What's with these weird BPWs on [insert model here]?
I specialize in optimized EXL2 quantization for models in the 70B to 100B+ range, specifically tailored for 48GB VRAM setups. My rig is built using 2 x 3090s with a Ryzen APU (APU used solely for desktop output—no VRAM wasted on the 3090s). I use TabbyAPI for inference, targeting context sizes between 32K and 64K.
Every model I upload includes a config.yml file with my ideal TabbyAPI settings. If you're using my config, don’t forget to set PYTORCH_CUDA_ALLOC_CONF=backend:cudaMallocAsync to save some VRAM.