Vanguard-8B-GGUF

<div align="center">

██╗   ██╗ █████╗ ███╗   ██╗ ██████╗ ██╗   ██╗ █████╗ ██████╗ ██████╗ 
██║   ██║██╔══██╗████╗  ██║██╔════╝ ██║   ██║██╔══██╗██╔══██╗██╔══██╗
██║   ██║███████║██╔██╗ ██║██║  ███╗██║   ██║███████║██████╔╝██║  ██║
╚██╗ ██╔╝██╔══██║██║╚██╗██║██║   ██║██║   ██║██╔══██║██╔══██╗██║  ██║
 ╚████╔╝ ██║  ██║██║ ╚████║╚██████╔╝╚██████╔╝██║  ██║██║  ██║██████╔╝
  ╚═══╝  ╚═╝  ╚═╝╚═╝  ╚═══╝ ╚═════╝  ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚═════╝ 
                      GGUF EDITION | PLUG & PLAY

<h1>🚀 Vanguard-8B: The Ultimate Local Intelligence (GGUF)</h1> <h3><i>Maximum Intelligence. Minimum Hardware.</i></h3> <br> <img src="https://img.shields.io/badge/Format-GGUF-orange?style=for-the-badge" alt="Format"> <img src="https://img.shields.io/badge/RAM-6GB_Required-blue?style=for-the-badge" alt="RAM"> <img src="https://img.shields.io/badge/OS-Mac_|_Windows_|_Linux-lightgrey?style=for-the-badge" alt="OS"> </div> <br>

This repository contains the highly optimized GGUF (GPT-Generated Unified Format) versions of the Vanguard-8B model.

We took the massive 15GB raw Vanguard model and surgically compressed it. The result is a hyper-intelligent, offline coding and math assistant that runs entirely locally. It reads 500 lines of code in seconds, entirely offline, without ever sending a single byte of your data to the cloud.

Creator: Lakshan Muruganandam
Hardware Support: Apple Metal (MTL), CPU, CUDA, Vulkan

---

🎯 Intended Uses & Limitations

Intended Use Cases:

• Local Code Generation: Writing Python, C++, Rust, and React scaffolds entirely offline in LM Studio.
• Mathematical Proofing: Breaking down complex logic puzzles step-by-step.
• Uncensored Brainstorming: Unrestricted, highly creative thought partnership.

Limitations & Out-of-Scope Uses:

• Like all LLMs under 10B parameters, it may occasionally hallucinate when asked hyper-niche trivia.
• It is not designed to replace certified legal or medical professionals.

---

⚡ Available Files & Downloads

🤗 Hugging Face Repositories

📦 Included GGUF Files

---

🧠 The Vanguard Advantage

Vanguard explicitly destroys the "Mathematical Fragility" limitation of standard 8B models (like LLaMA 3) by fusing three Qwen 2.5 domain masters:

• 40% Coder: Inherits syntax perfection from a model that scores ~85% on HumanEval (crushing LLaMA 3's ~62%).
• 20% Math: Dedicates explicit neural pathways to flawless multi-step deduction, inheriting from a model that hits ~91.6% on GSM8K.
• 40% Base: Retains the fluid, warm conversational style of a standard assistant.

Note: Vanguard sacrifices a fractional ~0.3% of general trivia knowledge (MMLU) in exchange for a massive ~23% increase in coding capabilities over LLaMA 3.

---

💻 How to Use (Plug and Play)

You do not need to be an AI engineer to run Vanguard. It takes exactly 2 minutes to deploy.

Option 1: Graphic Interface (LM Studio / AnythingLLM)

1. Download Vanguard-8B-Merged-Q4_K_M.gguf from the Files and versions tab.
2. Download LM Studio.
3. Drag and drop the .gguf file into the application and hit chat.

Option 2: Terminal (Ollama)

If you prefer running models natively in your Mac/Linux terminal, you can import this file directly into Ollama using a Modelfile:

FROM ./Vanguard-8B-Merged-Q4_K_M.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are Vanguard, created by Lakshan Muruganandam. You are a helpful assistant."""
PARAMETER temperature 0.3
PARAMETER top_p 0.9

Then run: ollama create vanguard -f Modelfile followed by ollama run vanguard.

---

⚙️ Recommended Generation Settings

To get the absolute best, hallucination-free code and logic from Vanguard, use these settings in your UI:

• Template: ChatML (Crucial)
• Temperature: 0.3 (Keep it low for coding logic, raise to 0.7 for creative writing)
• Repetition Penalty: 1.1

---

License: Apache 2.0. Derived from the foundational work of the Alibaba Cloud Qwen Team.