Instella-Prometheus-GGUF

Instella-Prometheus GGUF 🔥

The code-fire, packed for local inference.

This is the dedicated GGUF release of Instella-Prometheus, a merged, standalone coding model built from AMD's Instella-MoE-16B-A3B-SFT. Prometheus is a 16B-total / ~2.8B-active MoE model, distilled for direct, code-first answers under a bare user prompt:

• No system prompt required
• No thinking-tag suppression or decoder bans
• No LoRA adapter or PEFT dependency
• Designed for llama.cpp and compatible GGUF runtimes

Choose your quantization

All files are in the repository root so the Hugging Face Hub can identify and display them as GGUF quantized variants.

llama.cpp

# Example: Q4_K_M
./llama-cli \
  -m Instella-Prometheus-Q4_K_M.gguf \
  -p "Write a Python function to merge overlapping intervals." \
  -n 1024

For chat frontends, use the model's built-in GGUF metadata and chat template when supported by the runtime. The intended contract is simply: user task in, clean answer out.

What changed from base?

On a 12-task bare-user Python suite:

The full Transformers release, training details, dataset provenance, and evaluation notes are available in the canonical model repository.

License

Apache-2.0. See the canonical repository for base-model and dataset acknowledgements.

The fire belongs to them. The torch is yours.