Atlas-Coder-0.5B-GGUF

<div align="center">

⚡ Atlas-Coder-0.5B — GGUF

Quantized GGUF versions of Atlas-Coder-0.5B for CPU inference

</div>

---

About This Repository

This repository contains GGUF quantizations of Atlas-Coder-0.5B — a coding-specialized language model fine-tuned from Qwen2.5-Coder-0.5B base using QLoRA on 80K decontaminated code instructions.

These GGUF files are optimized for CPU inference on consumer hardware using llama.cpp, Ollama, and LM Studio. No GPU required.

For full training details, benchmark results, and the original FP16 model, see the Atlas-Coder-0.5B main model page.

---

Available Quantizations

Which Quantization Should I Use?

• Q8_0 → Maximum fidelity, RAM not a concern, you want the closest to FP16 output
• Q6_K → Best all-rounder. This is the one to use if you are unsure
• Q5_K_M → Excellent quality at a smaller footprint. Barely distinguishable from Q6_K for most tasks
• Q4_K_M → Minimum RAM usage. Still surprisingly capable for a 400MB file

All four files run comfortably on any modern laptop with 4GB+ RAM.

---

Quick Start

Ollama

# Download the model file first, then create a Modelfile
cat > Modelfile << EOF
FROM ./Atlas-Coder-0.5B-Q5_K_M.gguf

SYSTEM "You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning."

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER num_predict 512
EOF

ollama create atlas-coder -f Modelfile
ollama run atlas-coder "Write a Python function to reverse a linked list"

LM Studio

1. Open LM Studio
2. Go to Search tab
3. Search for Siddh07ETH/Atlas-Coder-0.5B-GGUF
4. Download your preferred quantization
5. Load and chat

llama.cpp (CLI)

# Clone and build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Run inference
./llama-cli \
  -m Atlas-Coder-0.5B-Q5_K_M.gguf \
  -p "<|im_start|>system\nYou are Atlas-Coder, an elite AI coding assistant.<|im_end|>\n<|im_start|>user\nWrite a Python binary search function.<|im_end|>\n<|im_start|>assistant\n" \
  -n 512 \
  --temp 0.3 \
  --top-p 0.9 \
  --repeat-penalty 1.1 \
  --no-display-prompt

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="Atlas-Coder-0.5B-Q5_K_M.gguf",
    n_ctx=1024,
    n_threads=4,
    verbose=False,
)

prompt = """<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code.<|im_end|>
<|im_start|>user
Write a Python function to check if a number is prime.<|im_end|>
<|im_start|>assistant
"""

output = llm(
    prompt,
    max_tokens=512,
    temperature=0.3,
    top_p=0.9,
    repeat_penalty=1.1,
    stop=["<|im_end|>", "<|im_start|>"],
)

print(output["choices"][0]["text"])

---

Recommended Generation Settings

---

Chat Template

Atlas-Coder uses ChatML format. Always wrap prompts in this structure:

<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning.<|im_end|>
<|im_start|>user
YOUR QUESTION HERE<|im_end|>
<|im_start|>assistant

---

Original Model

This GGUF repository is derived from the full FP16 model:

➡️ Siddh07ETH/Atlas-Coder-0.5B

The original model page contains:

• Full training methodology and pipeline details
• Dataset mixture and decontamination details
• Benchmark results (EvalPlus / HumanEval+ — publishing soon)
• FP16 and 4-bit inference code examples
• Research context and engineering decisions

---

About Atlas-Coder-0.5B

Atlas-Coder-0.5B is a coding-specialized LLM instruction-tuned from Qwen2.5-Coder-0.5B base (not instruct) using QLoRA on a Tesla T4. Key training highlights:

• 80K samples across 4 code datasets, including 50K execution-verified OSS-Instruct samples
• n-gram Jaccard decontamination against HumanEval — benchmark scores are honest
• Response-only loss masking — gradients only on assistant code output
• Trained entirely on free Kaggle T4 GPU — reproducible on consumer hardware
• 3-layer checkpoint recovery — survived Kaggle's 12-hour session limit

Part of the Pluto AI research project by Siddharth N.R.

---

Performance

Benchmarks (EvalPlus / HumanEval+) are currently being run on the original FP16 model and will be published on the main model page shortly.

GGUF quantization quality vs FP16 baseline (approximate):

---

Author

Siddharth N.R. Final-year B.Tech — AI & Data Science Pluto AI Research

---

Related Models

---

Citation

@misc{atlascoder2026,
  author    = {Siddharth N.R.},
  title     = {Atlas-Coder-0.5B: A QLoRA-Trained Sub-1B Coding Model from Base},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Siddh07ETH/Atlas-Coder-0.5B}
}

---

License

Apache 2.0 — see LICENSE. Base model Qwen2.5-Coder-0.5B is also Apache 2.0.