Qwen2.5-3B-Instruct.Q5_spec_Dot_code_generation

Qwen2.5-3B-Instruct (LoRA DOT Fine-Tune, GGUF Q5)

A parameter-efficient fine-tune of Qwen2.5-3B-Instruct specialized for Graphviz DOT code generation.
This model was trained with LoRA adapters and exported in GGUF Q5_K_M quantization for lightweight, portable inference.

---

📌 Model Summary

• Base Model: Qwen2.5-3B-Instruct (bnb-4bit)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Export Format: GGUF (llama.cpp-compatible)
• Quantization: Q5_K_M (5-bit, medium precision)
• File Size: 2.22 GB
• License: Apache 2.0
• Task: Natural language → Graphviz DOT code
• Hosting: Hugging Face Hub

---

🚀 Motivation

The base Qwen2.5-3B-Instruct model often produced:

• ❌ Syntax errors (malformed nodes/edges, unclosed brackets, invalid attributes)
• ❌ Hallucinated content (extra nodes/edges not in the prompt)

Fine-tuning with LoRA adapters on a curated dataset of 671 compiler-validated DOT examples resolved these issues, ensuring structurally valid DOT syntax generation.

---

📂 Training Details

• Dataset Source: Graphviz Gallery (paired natural-language prompts + DOT code)
• Validation: Every DOT sample compiled successfully before inclusion
• Final Corpus: ~671 bug-free instruction/output pairs
• Format: Alpaca-style JSON (train.jsonl, val.jsonl)
• Trainer: Hugging Face trl.SFTTrainer
• Environment: Google Colab GPU runtime

---

⚙️ Technical Stack

• Unsloth – optimized fine-tuning framework
• Unsloth Zoo – pretrained configs + utilities
• Tokenizer Utils – fixes for DOT-specific symbols
• TRL – supervised fine-tuning trainer
• Causal Convid + Mamba SSM – efficient sequence modeling for long DOT scripts
• Ninja + shutil – build + file management utilities

---

📦 Repository Contents

• Qwen2.5-3B-Instruct.Q5_K_M.gguf (2.22 GB) – quantized model weights
• README.md – model card
• .gitattributes – LFS configuration

---

🖥️ Inference & Deployment

Compatible runtimes:

• llama.cpp (CLI / server)
• llama-cpp-python
• Ollama
• LM Studio, Jan, Docker runners
• Unsloth Studio, Lemonade

Example (llama.cpp CLI):

./main -m Qwen2.5-3B-Instruct.Q5_K_M.gguf -p "Generate a DOT diagram for a binary tree"