Qwen2.5-1.5B-Instruct-Code-En-GGUF-789MB

Qwen2.5-1.5B-Instruct-Extreme-Compression (789 MB GGUF)

This is an extremely optimized, lightweight version of Qwen2.5-1.5B-Instruct, compressed down to just 789 MB in GGUF format (INT4 quantization).

To achieve this ultra-compact size, advanced optimization techniques were applied, including heavy pruning of non-essential language components and embedding reduction. The model's primary focus has been narrowed down to English language proficiency and Python programming/scripting tasks, while completely preserving its conversational, reasoning, and chain-of-thought ("thinking") capabilities.

🚀 Key Features

• Ultra-Lightweight: Compressed to only 789 MB—ideal for mobile devices, low-spec hardware, and edge computing.
• Specialized Focus: Retains high performance in English text generation and Python coding/scripting tasks.
• Intact Reasoning: Conversational logic, structural chat formats, and reasoning capabilities were left untouched during the pruning process.
• Format: GGUF (4-bit quantization), fully ready for local deployment.

🛠️ How to Use

1. LM Studio / AnythingLLM / KoboldCPP

1. Download the .gguf file from this repository.
2. Load the file directly into LM Studio or your preferred GGUF-compatible local GUI.
3. Set the system prompt and use the standard Qwen template.

2. Python via llama-cpp-python

from llama_cpp import Llama

1. Initialize the model (Make sure 'model_q4_k_m.gguf' is in the same directory)

llm = Llama( model_path="./model_q4_k_m.gguf", n_ctx=2048, # Context window size n_threads=4, # Number of CPU threads (adjust based on your CPU) n_gpu_layers=0 # Increase if you want to offload layers to GPU )

2. Prepare the prompt using the standard ChatML / Qwen template

system_prompt = "You are a helpful AI assistant specialized in English and Python coding." user_prompt = "Write a clean Python script to scrape data from a website using requests and BeautifulSoup."

full_prompt = ( f"<|im_start|>system\n{system_prompt}<|im_end|>\n" f"<|im_start|>user\n{user_prompt}<|im_end|>\n" f"<|im_start|>assistant\n" )

3. Run generation

print("\n--- Generating Response ---") output = llm( prompt=full_prompt, max_tokens=1024, temperature=0.7, stop=["<|im_end|>", "<|im_start|>"], # Crucial stop tokens for GGUF format echo=False )

4. Print the clean output

print(output["choices"][0]["text"].strip())