Downloads · 30 days
123
25% of all-time downloads
G4-MeroMero-26B-A4B-GGUF-ROCMFPX
JackBinary/G4-MeroMero-26B-A4B-GGUF-ROCMFPX
G4-MeroMero-26B-A4B-GGUF-ROCMFPX is a machine learning model from JackBinary. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a quantization of llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic (a Heretic-abliterated version of zerofata/G4-MeroMero-26B-A4B, a Gemma4-architecture 26B MoE: 128 experts, top-8, ~4B activated).
All-time downloads
498
Public
Repo size
17 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf17 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Aug 14, 2026
- Updated
- Aug 19, 2026
- SHA
- 8796afa6
Base models
Model card
From the Hugging Face model README
G4-MeroMero-26B-A4B (uncensored heretic) — ROCmFPX hybrid quant
This is a quantization of llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic
(a Heretic-abliterated version of zerofata/G4-MeroMero-26B-A4B,
a Gemma4-architecture 26B MoE: 128 experts, top-8, ~4B activated).
[!IMPORTANT] You need the ROCmFPX fork of llama.cpp (or a llama.cpp build with ROCmFPX support). This file uses the experimental
q4_0_rocmfp4_fast(type 101) andq8_0_rocmfpx(type 103) weight formats, which stock llama.cpp releases do not understand — loading it elsewhere will fail with an unknown tensor type error.
[!NOTE] The main quant is text-only; the original model is multimodal (Gemma4 vision tower). This repo includes vision projectors exported from the same source weights:
G4-MeroMero-26B-A4B-mmproj-F16.gguf(F16) andG4-MeroMero-26B-A4B-mmproj-BF16.gguf(BF16), for image input support.
Recipe
| Tensor group | Type | Notes |
|---|---|---|
Routed experts: blk.N.ffn_gate_up_exps (fused gate+up), blk.N.ffn_down_exps | q4_0_rocmfp4_fast (4.25 bpw) | Gemma4 packs gate/up into one fused tensor |
| Everything else quantizable (attention, router, embeddings) | q8_0_rocmfpx (8.25 bpw) | |
Norms, biases, per-expert scales (*.scale), router scales | f32 (untouched) |
- Total size: ~14.0 GB (4.64 bpw) vs 50.5 GB BF16 intermediate
How it was made
# 1. HF weights -> BF16 GGUF (from the ROCmFPX fork)
python3 convert_hf_to_gguf.py G4-MeroMero --outfile G4-MeroMero-26B-A4B-BF16.gguf --outtype bf16
# vision projector, exported from the same source weights (add --outtype f16 for the F16 variant):
python3 convert_hf_to_gguf.py G4-MeroMero --mmproj --outfile G4-MeroMero-26B-A4B-mmproj-BF16.gguf
# 2. hybrid quant (CPU-only build works fine for quantization)
llama-quantize \
--tensor-type "ffn_(gate_up|down)_exps.weight=q4_0_rocmfp4_fast" \
G4-MeroMero-26B-A4B-BF16.gguf \
G4-MeroMero-26B-A4B-Q8_0_ROCMFPX-Q4FAST-experts.gguf Q8_0_ROCMFPX
Usage
# build ROCmFPX for your GPU (see the repo README; e.g. Strix Halo):
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh
./build-strix-rocmfp4/bin/llama-cli \
-m G4-MeroMero-26B-A4B-Q8_0_ROCMFPX-Q4FAST-experts.gguf \
--mmproj G4-MeroMero-26B-A4B-mmproj-F16.gguf \
-dev Vulkan0 -ngl 999 -fa on --jinja
Benchmarks
[!NOTE] All Benchmarks were run on an AI max 395+ with 128GB RAM
| Depth | pp512 t/s | tg128 t/s |
|---|---|---|
| 4096 | 1038.70 ± 6.14 | 42.91 ± 0.24 |
| 16384 | 763.98 ± 8.60 | 40.84 ± 0.10 |
| 65536 | 383.38 ± 5.45 | 34.79 ± 0.27 |
Quality comparison vs BF16 source (perplexity / KLD): TBD.
Attribution & license
- Quantized from:
llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic(safetensors, converted to GGUF in-house) - Base model:
zerofata/G4-MeroMero-26B-A4B; abliteration via Heretic - License: Apache-2.0 (inherited)
- Quant formats by the ROCmFPX project
- License
- apache-2.0
- Created
- Aug 14, 2026
- Updated
- Aug 19, 2026