mumble-cleanup-2stage-GGUF

mumble-cleanup-2stage — Q4_0 GGUF

A Q4_0 GGUF quantization of amitashwini/mumble-cleanup-2stage, a LoRA fine-tune of Qwen/Qwen2.5-0.5B-Instruct trained to clean up raw speech-to-text transcripts — removing filler words, false starts and self-corrections while preserving the speaker's meaning.

This repository exists because upstream publishes only f16 and Q4_K_M. It adds the missing Q4_0 build. Nothing here is a new model: same weights, different quantization target.

Why Q4_0 and not Q4_K_M

On-device latency. llama.cpp ships ARM i8mm / dotprod dot-product kernels that the legacy Q4_0 type hits directly; Q4_K_M does not benefit from that path.

Measured on a Pixel 10a in Ramblr:

That is a real-device measurement of one app's prompt and pipeline, not a general benchmark.

Files

Quantized from upstream's f16 GGUF, SHA-256 7659e5dc4df164b50be3dce70d80b191fe7ac378a9e8e44b92e5e4313ef9ff82.

Verify after download:

shasum -a 256 mumble-cleanup-2stage-q4_0.gguf

Reproducing this file

Produced with the stock llama-quantize from llama.cpp (no custom tooling, no hand-edited tensors), at submodule revision 152d337fa (tag b9867):

# 1. fetch upstream f16
huggingface-cli download amitashwini/mumble-cleanup-2stage \
  mumble-cleanup-2stage-f16.gguf --local-dir .

# 2. confirm the source before quantizing
shasum -a 256 mumble-cleanup-2stage-f16.gguf
# expect 7659e5dc4df164b50be3dce70d80b191fe7ac378a9e8e44b92e5e4313ef9ff82

# 3. quantize
llama-quantize mumble-cleanup-2stage-f16.gguf \
  mumble-cleanup-2stage-q4_0.gguf Q4_0

Prompting — important

This is a fine-tune, not a general instruct model being asked nicely. It was trained on one exact system-prompt string and should be given that string verbatim. Do not template it, do not interpolate a vocabulary list into it, and do not append instructions — those degrade output, because the model is pattern-matching against its training format rather than following instructions in the general sense.

Its two-stage output format is also worth checking against your parser before you wire it in.

Quality

In an offline A/B against LFM2.5-350M on a purpose-built transcript corpus, each model run with its own production prompt, this model scored 80.6% vs 66.3%. Single-corpus, single-run — treat it as directional, not as a leaderboard claim.

License

Apache-2.0, inherited from both the upstream fine-tune and the Qwen2.5-0.5B-Instruct base. Quantization does not create a new license; all credit for the weights belongs to amitashwini and the Qwen team.