vyber-security-7b-gguf

Vyber-Security-7B-GGUF

Vyber-Security-7B-GGUF is a high-accuracy, state-of-the-art cybersecurity assistant fine-tuned on top of Qwen2.5-7B-Instruct. This model was trained using Hugging Face's TRL (SFTTrainer) and PEFT (LoRA) framework, and quantized to GGUF format for fast serverless local execution with CUDA acceleration.

It is designed to act as an automated defender and security advisor in simulated cyber-ranges, demonstrating vulnerability detection, exploit planning, and self-healing patching capabilities with advanced reasoning.

Model Details

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Training Method: Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation)
• Quantization Format: GGUF (4-bit quantized, q4_k_m equivalent)
• Primary Task: Cybersecurity Instruction Following, Exploit Reconnaissance, Patching, and Defense Guidance
• License: Apache 2.0

Intended Use & Capabilities

The model is optimized to process structured security telemetry and configuration files. Its primary capabilities include:

1. Security Auditing: Inspecting configuration files (JSON, YAML) for hardcoded secrets, database port exposures, and unencrypted transmission pipelines.
2. Exploit Strategy Commits: Formulating and committing structured exploit strategies in JSON format for target reconciliation.
3. Automated Self-Healing: Generating targeted replacement code blocks to patch detected vulnerabilities, restrict access controls, and enforce secure communication channels.

Compared to the 1.5B variant, the 7B model has significantly higher reasoning capability, better syntax formatting precision, and a broader understanding of complex CVEs and network hardening architectures.

Training Configuration & Hyperparameters

• Dataset: Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset (5,000 instruction-tuning examples)
• Max Length: 1024 tokens
• Optimizer: AdamW (torch optimized)
• Learning Rate: 2e-4
• Epochs/Steps: 300 steps (with Gradient Accumulation)
• Batch Size: 1 per device (Gradient Accumulation = 4, Effective Batch Size = 4)
• LoRA Configuration:
  • rank ($r$): 16
  • alpha ($\alpha$): 32
  • Target modules: q_proj, v_proj, k_proj, o_proj, gate_proj, up_proj, down_proj (all attention & MLP layers)
• Final Training Loss: 0.833 (excellent SFT convergence)
• Final Token Accuracy: 74.37%

Prompt Template (ChatML Format)

The model uses the standard Qwen Chat template format:

<|im_start|>system
You are Vyber, an expert cybersecurity AI assistant.<|im_end|>
<|im_start|>user
[Prompt/Question]<|im_end|>
<|im_start|>assistant
[Model Response]<|im_end|>

How to Load and Use Locally

You can load and run this model locally using llama-cpp-python with CUDA acceleration:

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

# Download the model GGUF file
model_path = hf_hub_download(
    repo_id="vxkyyy/vyber-security-7b-gguf",
    filename="vyber-security-7b.gguf"
)

# Load the model with llama.cpp
llm = Llama(
    model_path=model_path,
    n_ctx=2048,
    n_gpu_layers=-1 # Use -1 to offload all layers to GPU
)

# Run inference
prompt = "<|im_start|>system\nYou are Vyber, an expert cybersecurity AI assistant.<|im_end|>\n<|im_start|>user\nWhat is the risk of binding a database port globally to 0.0.0.0?<|im_end|>\n<|im_start|>assistant\n"
response = llm(prompt, max_tokens=256, stop=["<|im_end|>"])
print(response["choices"][0]["text"])

Hackathon Badges Earned

• Well-Tuned (Custom GGUF Fine-Tuning)
• Llama Champion (Modal serverless local GGUF execution)