Muse-Glimmer-30B-autoscheme-3.5bit-XS

kaitchup/Muse-Glimmer-30B-autoscheme-3.5bit-XS

Muse-Glimmer-30B-autoscheme-3.5bit-XS is a machine learning model from kaitchup. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

<div align="center" <img src="https://cdn-uploads.huggingface.co/production/uploads/64b93e6bd6c468ac7536607e/mj6xac74jHGLqymiovObc.png" alt="The Kaitchup -- AI on a Budget" style="width: 100%; max-width: 100%; height:…

Downloads · 30 days

28% of all-time downloads

All-time downloads

151

Public

Parameters

6.3B

18.2 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors18.2 GB · 100%

Parameter types

How the weights are stored.

I3227B · 89%

Base models

meta-models/Muse-Glimmer-30B

Model card

From the Hugging Face model README

<div align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/64b93e6bd6c468ac7536607e/mj6xac74jHGLqymiovObc.png" alt="The Kaitchup -- AI on a Budget" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" /> <div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;"> <a href="https://kaitchup.substack.com/subscribe"><strong>Subscribe and Support</strong></a> </div> </div>

This is meta-models/Muse-Glimmer-30B quantized with AutoRound using a target bitwidth set at 3.5 and a 4-bit LM head. Model packed by llm-compressor.

The model is compatible with vLLM (tested: v0.26+muse glimmer). Made and Tested with an RTX Pro 6000 (Google Colab's G4). Currently under evaluation: Subscribe to be notified when I publish the results.

Made by: The Kaitchup

Instructions

docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:muse-glimmer kaitchup/Muse-Glimmer-30B-autoscheme-3.5bit-XS \
  --generation-config auto \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser muse_glimmer \
  --reasoning-parser muse_glimmer

Type: muse_glimmer
License: apache-2.0
Created: Aug 11, 2026
Updated: Aug 11, 2026