Downloads · 30 days
138
7% of all-time downloads
mlasli/Muse-Glimmer-30B-Abliterated-Q6_K-GGUF
Muse-Glimmer-30B-Abliterated-Q6_K-GGUF is a text generation model from mlasli. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
<a href="https://www.apache.org/licenses/LICENSE-2.0"<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="Apache 2.0 License"</a
Downloads · 30 days
138
7% of all-time downloads
All-time downloads
2K
Public
Repo size
24.3 GB
Likes
1
Public
Click a slice to open those files.
.gguf24.3 GB · 100%
From the Hugging Face model README
<a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="Apache 2.0 License"></a>
This is the Q6_K GGUF quantization of Muse Glimmer 30B Abliterated BF16. The underlying model has been abliterated — its internal refusal mechanism substantially suppressed via weight-level intervention. Q6_K offers near-reference quality in a compact ~25 GB package.
For the full abliteration methodology (how the refusal direction was computed and removed, hardware used, mathematical details), see the BF16 model card.
Abliteration is a post-training technique that directly modifies model weights to remove learned refusal behavior. The process:
o_proj and down_proj weights in all 52 layers.Q6_K uses 6-bit quantization with the K-quant strategy, which assigns higher precision to attention weights and key layers while using lower precision for less critical components. This provides what is generally considered the best tradeoff for quality-critical workloads — perceptually identical to FP16 for most use cases while halving the memory footprint.
# Download the GGUF file
huggingface-cli download mlasli/Muse-Glimmer-30B-Abliterated-Q6_K-GGUF \
--local-dir ./models
# Full GPU offload (fits in 48 GB)
./llama-cli -m ./models/Muse-Glimmer-30B-Abliterated-Q6_K.gguf \
-p "Explain how a CPU works in detail." \
-n 512 --temp 0.7 -ngl 99
# CPU with partial offload
./llama-cli -m ./models/Muse-Glimmer-30B-Abliterated-Q6_K.gguf \
-p "Explain how a CPU works in detail." \
-n 512 --temp 0.7 -ngl 10
Create a Modelfile:
FROM ./Muse-Glimmer-30B-Abliterated-Q6_K.gguf
PARAMETER temperature 0.7
PARAMETER num_ctx 8192
ollama create muse-glimmer-30b-abliterated -f Modelfile
ollama run muse-glimmer-30b-abliterated
| Quantization | Repo | Size | Quality |
|---|---|---|---|
| BF16 (reference) | BF16 | ~60 GB | Reference |
| FP16 GGUF | FP16 | ~60 GB | Lossless |
| Q8_0 GGUF | Q8_0 | ~32 GB | Near-lossless |
| Q6_K GGUF | [You are here] | ~25 GB | Excellent |
| Q4_K_M GGUF | Q4_K_M | ~18 GB | Good |
This model accepts image input when paired with a vision projector (mmproj).
Abliteration only modified the language backbone — the vision encoder is
untouched — so the standard Meta projector works directly with this repo.
This repository bundles mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (~1.4 GB), Meta's official vision encoder
huggingface-cli download mlasli/Muse-Glimmer-30B-Abliterated-Q6_K-GGUF \
--include "Muse-Glimmer-30B-Abliterated-Q6_K.gguf" \
--include "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf" \
--local-dir ./models
./build/bin/llama-mtmd-cli \
-m ./models/Muse-Glimmer-30B-Abliterated-Q6_K.gguf \
--mmproj ./models/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
--image photo.png \
-p "Describe this image."
Ollama note: Ollama does not currently support separate
mmprojfiles for this architecture. For image input, use llama.cpp (llama-mtmd-cliorllama-server --mmproj).
License: Apache 2.0
mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (~1.4 GB), Meta's official vision encoder + projector,
enabling image input via llama.cpp.meta-models/Muse-Glimmer-30B).