magnum-72b-v1-llamaify-GGUF

magnum-72b-v1-llamaify

This is a converted version of the Magnum 72B v1 model, now in LLaMA format. The original model was designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus. This converted version maintains the same capabilities but is now compatible with LLaMA-based frameworks and tools.

The speed may also be a bit faster, especially if you use frameworks optimized for LLaMA.

Model Details

• Base Model: Qwen-2 72B Instruct
• Training Data: 55 million tokens of high-quality RP data
• Training Duration: 1.5 epochs
• Hardware Used: 8x AMD Instinct™ MI300X Accelerators

Context length is reduced to 32k, not sure how the sliding window implementation should be translated (afaik LLaMA doesn't use this).

Prompting

The model uses ChatML formatting for instructions. A typical input would look like this:

<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant

Credits

Credit goes to Anthracite for the original model.

Conversion Details

This version of the model has been converted to the LLaMA format to enhance compatibility with a wider range of tools and frameworks. While the core capabilities of the model remain the same, users should be aware that there might be slight differences in performance due to the conversion process.

Usage

Can be used in transformers or any software that supports LLaMA arch models.

You can download GGUF quantizations here: leafspark/magnum-72b-v1-llamaify-GGUF

Limitations

Users should be aware that while this converted model maintains the general capabilities of the original, there might be subtle differences in performance or behavior due to the format change. It's recommended to test the model for your specific use case.

License

This model inherits the license from its base model, Qwen-2 72B Instruct. Please refer to the original license for terms of use.

Contact

For questions or issues related to this converted model, please open an issue in the model's repository.