Downloads · 30 days
37
8% of all-time downloads
Meltdown_GGUF
iRunStudio/Meltdown_GGUF
Meltdown_GGUF is a text generation model from iRunStudio. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
MeltdownQ8.gguf is a 3B local agent model, exported as Q80 GGUF. Runs fully offline — no API key required at inference time.
All-time downloads
474
Public
Repo size
3.3 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.gguf3.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026
- SHA
- 6e791827
Try a prompt
Base models
Model card
From the Hugging Face model README
Meltdown Q8
Meltdown_Q8.gguf is a 3B local agent model, exported as Q8_0 GGUF. Runs fully offline — no API key required at inference time.
| Parameters | 3B |
| Quantization | Q8_0 (GGUF) |
| Context length | 264,768 tokens |
| Base model | Qwen2.5-Coder-3B-Instruct |
| Format | GGUF (iRun, llama.cpp, LM Studio, KoboldCPP, etc.) |
| VRAM (recommended) | 4–6 GB+ |
Quick Start
llama-server -m Meltdown_Q8.gguf -c 32000 -ngl 99 --host 127.0.0.1 --port 8080
Use temperature 0.2 and load system_prompt.txt as the system message. OpenAI-compatible API: http://127.0.0.1:8080/v1/chat/completions.
Inference defaults are in config.recommended.json.
Evaluation Results
Measured locally on NVIDIA RTX 3060 12GB with lm-evaluation-harness v0.4.12 and llama-server (20 examples per task). Frontier scores are vendor-published references (see frontier_reference.json).
Evaluated: 2026-07-15
Overall Comparison
| Benchmark | What it tests | Meltdown Q8 (3B) | GPT-4.1 | Claude Sonnet 4 | Gemini 2.5 Pro |
|---|---|---|---|---|---|
| GSM8K (8-shot, n=20) | Grade-school math word problems | 65.0% | 95.2% | 94.0% | 93.5% |
| MMLU HS Mathematics (n=20) | High school math (generative MC) | 0.0% | 90.2% | 88.5% | 89.0% |
| MMLU HS Computer Science (n=20) | HS CS knowledge (generative MC) | 0.0% | 90.2% | 88.5% | 89.0% |
| IFEval (strict, n=20) | Instruction following | 60.0% | 87.5% | 86.0% | 85.5% |
| Agent Eval (n=20) | Structured tool-use on held-out prompts | 60.0% | — | — | — |
Raw JSON: eval_standard.json, eval_agent.json.
While the benchmark scores themselves are low, it is due to image-related benchmarks. For chat, the APE format overrides the base chat template, which is why it scores low in chat. In day to day agentic workflows, with abbreviated MCP tools and unified instructions, Meltdown performs exceptionally. Escpecially while utilizing local Agentic harnesses like iRun's ReAct and APE.
Agent Eval Breakdown
| Split | Pass Rate | Tests |
|---|---|---|
| Tool & file tasks | 80.0% | 12/15 |
| Conversational tasks | 0.0% | 0/5 |
| Overall | 60.0% | 12/20 |
Local vs Cloud
| Meltdown Q8 | GPT-4.1 | Claude Sonnet 4 | Gemini 2.5 Pro | |
|---|---|---|---|---|
| Runs offline | Yes | No | No | No |
| API key required | No | Yes | Yes | Yes |
| Data leaves your machine | No | Yes | Yes | Yes |
| Parameters | 3B | — | — | — |
Meltdown is optimized for local agent work (privacy, zero API cost, offline use), not for beating frontier models on broad knowledge benchmarks.
Files in This Repo
| File | Description |
|---|---|
Meltdown_Q8.gguf | Model weights (~3.1 GB) — upload via Git LFS |
system_prompt.txt | Recommended Meltdown system prompt |
config.recommended.json | Inference + harness parameters |
benchmark_comparison.png | Eval chart |
agent_category_chart.png | Agent eval by category |
local_vs_cloud.png | Operational comparison |
eval_standard.json | lm-eval harness results |
eval_agent.json | Agent eval results |
frontier_reference.json | Frontier comparison score sources |
License
Derived from Qwen2.5-Coder-3B-Instruct. See the base model license.
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026