Qwen2.5-3B-CyberSecurity-Instruct

Qwen2.5-3B-CyberSec-Instruct

A fine-tuned version of Qwen2.5-3B-Instruct specifically designed for advanced cybersecurity analysis. This model is built to bridge the gap between high-level vulnerability descriptions and low-level exploit code execution.

• Developed by: Mohamedabul
• License: apache-2.0
• Finetuned from model: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
• Architecture: 3B parameters (4-bit QLoRA)

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

<!-- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth) -->

---

Model Capabilities

This LLM acts as an expert cybersecurity analyst and reverse engineer. It is capable of:

1. Vulnerability Triage: Automatically generating structured severity, attack vector, and mitigation reports for any CVE.
2. Exploit Reverse-Engineering: Analyzing raw exploit code (C, Python, Bash) to provide an immediate technical breakdown of how the exploit works and what vulnerabilities it targets.
3. Attack Chain Reasoning: Combining a CVE with raw exploit code to generate a step-by-step kill-chain analysis, from initial access to system compromise.

---

Training Data

To achieve maximum accuracy, the model was fine-tuned on an expansive historical corpus of modern vulnerabilities and exploits, completely uncapped and unfiltered:

• NVD CVE Database: Vulnerabilities published between 2020 through 2025.
• Exploit-DB: Over 45,000+ real-world exploits directly from Offensive Security.
• MITRE CWE: Full weakness classifications, likelihood of exploit, and abstractions.
• Total Dataset Size: ~187,700 structured instruction samples.

---

Evaluation Metrics

The fine-tuned model was evaluated against an unseen hold-out test dataset to mathematically verify its understanding of cybersecurity concepts and generation quality.

---

Usage & Inference

To load the model quickly using Unsloth for 2x faster inference:

from unsloth import FastLanguageModel

# Load the model directly from this Hugging Face repository
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Mohamedabul/Qwen2.5-3B-CyberSec-Instruct", # or your exact repo name
    max_seq_length=1024,
    dtype=None,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

# Example Prompt
instruction = "Analyze this vulnerability: CVE-2021-44228 (Log4Shell). Provide attack vectors, severity, and mitigation."
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": instruction}], tokenize=False, add_generation_prompt=True
)

inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])