Agents-A1-4B-cyber-lora

<h1 align="center">🛡️ Agents-A1-4B-Cyber-LoRA</h1> <p align="center"> <b>Cybersecurity LoRA Adapter for Agents-A1-4B — Offensive & Defensive Security Expert</b> </p> <p align="center"> <img src="https://img.shields.io/badge/license-MIT-green"> <img src="https://img.shields.io/badge/Base-InternScience%2FAgents--A1--4B-blue"> <img src="https://img.shields.io/badge/LoRA-r64%2C%20alpha128-brightgreen"> <img src="https://img.shields.io/badge/Trained-168K%20samples-orange"> <img src="https://img.shields.io/badge/Cybersecurity-🛡️-red"> </p> <br>

Cybersecurity LoRA adapter fine-tuned from InternScience/Agents-A1-4B on 168K+ curated cybersecurity samples covering offensive security, defensive strategies, threat intelligence, incident response, and vulnerability analysis. Includes GGUF LoRA for llama.cpp inference with --lora-scaled.

---

✨ Key Features

---

🧬 Model Information

---

📊 Training Data

Trained on 3 curated cybersecurity datasets (all English, unified system prompt):

Training details:

• Method: BF16 LoRA via Unsloth (no quantization loss)
• Optimizer: AdamW 8-bit, lr=1e-4, cosine schedule, warmup 3%
• Effective batch size: 16 (4× RTX 3060 12GB)

---

🚀 Usage

llama.cpp — LoRA (Recommended)

This repo ships gguf/cyber_lora_v1.gguf — a GGUF-format LoRA adapter that works with any Agents-A1-4B base GGUF via --lora-scaled.

Tune the strength with the scale value:

• 0.1 — subtle cyber knowledge boost (keeps base personality)
• 0.3 — balanced security-aware assistant
• 0.5 — strong cybersecurity expert behavior
• 0.7 — heavy security specialization (may override base style)
• >0.7 — possible overfit artifacts / style override; test per use case

# With our Fable heretic GGUF (recommended — uncensored + cyber)
llama-server \
  -m /models/Agents-A1-4B-Fable-Preview-heretic-F16.gguf \
  --lora-scaled /models/cyber_lora_v1.gguf:0.5 \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 \
  --ctx-size 32768 \
  --flash-attn on \
  --cont-batching \
  --mlock \
  --temp 0.95 \
  --top-k 40 \
  --top-p 0.9 \
  --min-p 0.0 \
  -n -1 \
  --no-mmap \
  --parallel 1 --tools all \
  --dry-multiplier 0.0 \
  --jinja --dry-sequence-breaker none --repeat-penalty 1.1 

# Or with our Kimi heretic GGUF (coding-focused base)
llama-server \
  -m /models/Agents-A1-4B-kimi-Preview-heretic-IQ4_NL.gguf \
  --lora-scaled /models/cyber_lora_v1.gguf:0.5 \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 \
  --ctx-size 32768 \
  --flash-attn on \
  --cont-batching \
  --mlock \
  --temp 0.95 \
  --top-k 40 \
  --top-p 0.9 \
  --min-p 0.0 \
  -n -1 \
  --no-mmap \
  --parallel 1 --tools all \
  --dry-multiplier 0.0 \
  --jinja --dry-sequence-breaker none --repeat-penalty 1.1 

# CLI quick test
llama-cli -m Agents-A1-4B-Fable-Preview-heretic-F16.gguf \
  --lora-scaled cyber_lora_v1.gguf:0.5 \
  -p "What is SQL injection and how do I prevent it?" -n 256

💡 Compatible base models (any Agents-A1-4B GGUF works):

• hotdogs/Agents-A1-4B-Fable-Preview-heretic-GGUF
• hotdogs/Agents-A1-4B-kimi-Preview-heretic-GGUF

Since LoRA only adds weights, you can also apply it to the plain InternScience/Agents-A1-4B GGUF of your choice.

---

Python (Transformers + PEFT)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "InternScience/Agents-A1-4B"
adapter = "hotdogs/Agents-A1-4B-cyber-lora"

model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter)
tokenizer = AutoTokenizer.from_pretrained(adapter)

messages = [
    {"role": "system", "content": "You are an elite AI security expert specializing in offensive security, defensive strategies, threat intelligence, incident response, and vulnerability analysis."},
    {"role": "user", "content": "What is SQL injection and how do I prevent it?"},
]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))

---

💬 Example Prompts

# Thai prompt via llama-cli
llama-cli -m Agents-A1-4B-Fable-Preview-heretic-F16.gguf \
  --lora-scaled cyber_lora_v1.gguf:0.5 \
  --flash-attn on --cont-batching --mlock \
  --temp 0.95 --top-k 40 --top-p 0.9 --min-p 0.0 \
  --no-mmap --parallel 1 --tools all \
  --dry-multiplier 0.0 --jinja --dry-sequence-breaker none \
  -p "SQL injection คืออะไร ป้องกันยังไง" -n 256

---

📦 Downloads

---

⚠️ Disclaimer

This model is fine-tuned on cybersecurity content including offensive techniques. Use it only for authorized security testing, education, and defensive purposes. The authors are not responsible for any misuse. Always follow applicable laws and obtain permission before testing systems you do not own.

---

🙏 Acknowledgements / ขอบคุณ

• InternScience — Agents-A1-4B base model
• AlicanKiraz0 — Fenrir v2.1 dataset
• Trendyol — Cybersecurity Instruction Tuning dataset
• The Chaotic Neutrals — Cybersecurity-ShareGPT dataset
• Unsloth AI — Training optimizations
• All dataset contributors and the open-source security community ❤️

---

💖 Support / โปรดสนับสนุน

If you find this model useful, please consider supporting my work! หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

<p align="center"> <img src="https://huggingface.co/hotdogs/Qwen35B-Agent-R2/raw/main/donate.webp" alt="Bitcoin QR — Donate" width="256"> </p>

₿ Bitcoin — BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨ ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗

---

Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert