Downloads · 30 days
41
2% of all-time downloads
RichardErkhov/anthracite-org_-_magnum-v2-72b-gguf
anthracite-org_-_magnum-v2-72b-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
41
2% of all-time downloads
All-time downloads
1.9K
Public
Repo size
857 GB
Likes
0
Public
Click a slice to open those files.
.gguf996 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
magnum-v2-72b - GGUF
| Name | Quant method | Size |
|---|---|---|
| magnum-v2-72b.Q2_K.gguf | Q2_K | 27.76GB |
| magnum-v2-72b.IQ3_XS.gguf | IQ3_XS | 30.59GB |
| magnum-v2-72b.IQ3_S.gguf | IQ3_S | 32.12GB |
| magnum-v2-72b.Q3_K_S.gguf | Q3_K_S | 32.12GB |
| magnum-v2-72b.IQ3_M.gguf | IQ3_M | 33.07GB |
| magnum-v2-72b.Q3_K.gguf | Q3_K | 35.11GB |
| magnum-v2-72b.Q3_K_M.gguf | Q3_K_M | 35.11GB |
| magnum-v2-72b.Q3_K_L.gguf | Q3_K_L | 36.79GB |
| magnum-v2-72b.IQ4_XS.gguf | IQ4_XS | 37.4GB |
| magnum-v2-72b.Q4_0.gguf | Q4_0 | 38.4GB |
| magnum-v2-72b.IQ4_NL.gguf | IQ4_NL | 38.9GB |
| magnum-v2-72b.Q4_K_S.gguf | Q4_K_S | 40.88GB |
| magnum-v2-72b.Q4_K.gguf | Q4_K | 44.16GB |
| magnum-v2-72b.Q4_K_M.gguf | Q4_K_M | 44.16GB |
| magnum-v2-72b.Q4_1.gguf | Q4_1 | 42.56GB |
| magnum-v2-72b.Q5_0.gguf | Q5_0 | 46.72GB |
| magnum-v2-72b.Q5_K_S.gguf | Q5_K_S | 47.85GB |
| magnum-v2-72b.Q5_K.gguf | Q5_K | 50.71GB |
| magnum-v2-72b.Q5_K_M.gguf | Q5_K_M | 50.71GB |
| magnum-v2-72b.Q5_1.gguf | Q5_1 | 50.88GB |
| magnum-v2-72b.Q6_K.gguf | Q6_K | 59.93GB |
| magnum-v2-72b.Q8_0.gguf | Q8_0 | 71.96GB |
license: other license_name: tongyi-qianwen license_link: https://huggingface.co/anthracite-org/magnum-v2-72b/blob/main/LICENSE base_model: Qwen/Qwen2-72B-Instruct language:

This is the seventh (Lucky!) in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus. This model is fine-tuned on top of Qwen-2 72B Instruct.
Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:
"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""
This model has been a team effort, and the credits goes to all members of Anthracite.
The training was done for 2 epochs. We used 8x AMD Instinct™ MI300X Accelerators for the full-parameter fine-tuning of the model.
We also trained with a weight decay of 0.01 to help further stabilize the loss trajectory and mitigate catastrophic forgetting, and utilize a peak learning rate of 4e-6 to prevent the 2nd epoch loss from dropping too significantly (as it is a strong indicator of overfitting).

Sample Packing was done for 16k tokens rather than the 8k tokens used in our previous runs.
...