Downloads · 30 days
0
Meta-i-hardly-know-er/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-MTP-4.0-bpw
Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-MTP-4.0-bpw is a machine learning model from Meta-i-hardly-know-er. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc.
Thank you to the DavidAU team for this finetune.
Downloads · 30 days
0
Access
Public
Updated Sep 3, 2026
Repo size
16.9 GB
Likes
1
Public
Click a slice to open those files.
.safetensors16.9 GB · 100%
From the Hugging Face model README
Thank you to the DavidAU team for this finetune.
4.0 bpw exllamav3 quant, head size is 8, avg 4.02 bpw across all layers.
gets about 10% better tokens per sec generation speeds on my nvidia setup using tp and mtp (min draft 1, max draft 2) compared to Q4 llama.cpp gguf. 40+ exllama vs 36+ llama.cpp
to note, the mmproj was not loaded during testing. both were loaded at 100,000 ctx. numbers from benchmarking using 2x 3060 rtx. real world factors may vary speeds.
during actual use: (brainstorming ideas)
the exllama quant settled at 32-40+ with up to 50 generation tps and 510-550 tps processing. llama.cpp settled at 29-36tps with up to 42, and kept 550-600+ tps processing speed.
little less -(8-19%) prompt processing though compared to llama.cpp.
made this because i prefill a lot of context, so processing speed isnt as important to me.
also, bonus, uses much less vram comparatively.
llama.cpp:

exllamav3:
