Downloads · 30 days
42
4% of all-time downloads
Serpen/Minimax_M3_MMPROJ_GGUF
Minimax_M3_MMPROJ_GGUF is a machine learning model from Serpen. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
These are mmproj files for running Minimax-M3 with multi modal contexts in llama.cpp. Requires this pr: https://github.com/ggml-org/llama.cpp/pull/25113. It will NOT load with Unsloth's or the non vision MSA PR. Furth…
Downloads · 30 days
42
4% of all-time downloads
All-time downloads
992
Public
Repo size
3.5 GB
Likes
0
Public
Click a slice to open those files.
.gguf1.7 GB · 100%
From the Hugging Face model README
These are mmproj files for running Minimax-M3 with multi modal contexts in llama.cpp. Requires this pr: https://github.com/ggml-org/llama.cpp/pull/25113. It will NOT load with Unsloth's or the non vision MSA PR. Furthermore the linked PR depends on Minimax Sparse Attention, so Unsloth's GGUFs will NOT load on it. You will need to convert or download GGUFs with the Indexer tensors preserved: https://huggingface.co/Serpen/Minimax-M3-MSA-GGUF. Both Vision and MSA are fully functional, at low ctx decoding speed is around the same as Unsloth's PR for now, but MSA starts pulling ahead at longer context sizes.
What MSA helps with then you may ask?
ELI5: the model was trained to only ever show 2048 tokens of ctx to any single indexer head. (Chosen individually for each of the 4 indexer heads for every sparse layer). This is what the model was trained on. Deviating from this will cause long context recall issues. Unsloth's PR instead shows the full context to every single one of these heads. This dilutes attention, and pushes the model out of distribution.