Downloads · 30 days
445
3% of all-time downloads
mlasli/Muse-Glimmer-30B-Abliterated-Q4_K_M-GGUF
Muse-Glimmer-30B-Abliterated-Q4_K_M-GGUF is a text generation model from mlasli. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
<a href="https://www.apache.org/licenses/LICENSE-2.0"<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="Apache 2.0 License"</a
Downloads · 30 days
445
3% of all-time downloads
All-time downloads
14.2K
Public
Repo size
18.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf18.3 GB · 100%
From the Hugging Face model README
<a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="Apache 2.0 License"></a>
This is the Q4_K_M GGUF quantization of Muse Glimmer 30B Abliterated BF16. The underlying model has been abliterated — its internal refusal mechanism substantially suppressed via weight-level intervention. The Q4_K_M quant offers the best balance between size and quality for consumer hardware, fitting in ~18 GB of VRAM/RAM.
For the full abliteration methodology (how the refusal direction was computed and removed, hardware used, mathematical details), see the BF16 model card.
Abliteration is a post-training technique that directly modifies model weights to remove learned refusal behavior. The process:
o_proj and down_proj weights in all 52 layers.Q4_K_M uses a 4-bit quantization with a medium-sized key-value cache quantization. This is the recommended quant for most users — it achieves excellent quality while being compact enough for a single 24 GB GPU (RTX 3090/4090) or split across dual 12 GB GPUs. Model weights are quantized with a block size that carefully preserves outlier weights, and the K-quant strategy applies separate quantization precision to different weight types (attention vs. MLP).
# Download the GGUF file
huggingface-cli download mlasli/Muse-Glimmer-30B-Abliterated-Q4_K_M-GGUF \
--local-dir ./models
# CPU-only inference
./llama-cli -m ./models/Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf \
-p "Explain how a CPU works in detail." \
-n 512 --temp 0.7 -ngl 0
# GPU offload (20 layers to GPU for 24 GB cards)
./llama-cli -m ./models/Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf \
-p "Explain how a CPU works in detail." \
-n 512 --temp 0.7 -ngl 20
Create a Modelfile:
FROM ./Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER num_ctx 8192
ollama create muse-glimmer-30b-abliterated -f Modelfile
ollama run muse-glimmer-30b-abliterated
| Quantization | Repo | Size | Quality |
|---|---|---|---|
| BF16 (reference) | BF16 | ~60 GB | Reference |
| FP16 GGUF | FP16 | ~60 GB | Lossless |
| Q8_0 GGUF | Q8_0 | ~32 GB | Near-lossless |
| Q6_K GGUF | Q6_K | ~25 GB | Excellent |
| Q4_K_M GGUF | [You are here] | ~18 GB | Good |
This model accepts image input when paired with a vision projector (mmproj).
Abliteration only modified the language backbone — the vision encoder is
untouched — so the standard Meta projector works directly with this repo.
This repository bundles mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (~1.4 GB), Meta's official vision encoder
huggingface-cli download mlasli/Muse-Glimmer-30B-Abliterated-Q4_K_M-GGUF \
--include "Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf" \
--include "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf" \
--local-dir ./models
./build/bin/llama-mtmd-cli \
-m ./models/Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf \
--mmproj ./models/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
--image photo.png \
-p "Describe this image."
Ollama note: Ollama does not currently support separate
mmprojfiles for this architecture. For image input, use llama.cpp (llama-mtmd-cliorllama-server --mmproj).
License: Apache 2.0
mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (~1.4 GB), Meta's official vision encoder + projector,
enabling image input via llama.cpp.meta-models/Muse-Glimmer-30B).