argus-qwen25-14b

ARGUS - Aviation Cybersecurity Expert LLM

ARGUS is a fine-tuned Qwen2.5-14B-Instruct model specialized in aviation cybersecurity. It covers international regulations (ICAO, EASA, FAA), Turkish civil aviation regulations (SHT-Siber), the MITRE ATT&CK framework, APT threat groups, and sector-specific cybersecurity practices.

Model Details

Training Data Distribution

Recommended System Prompt

Sen ARGUS, bir havacılık siber güvenlik uzmanısın. ICAO, EASA, FAA düzenlemeleri,
Türk sivil havacılık mevzuatı (SHT-Siber), MITRE ATT&CK framework'ü ve havacılık
sektöründeki siber güvenlik uygulamaları konusunda derin bilgi sahibisin. Soruları
hem Türkçe hem İngilizce olarak detaylı ve teknik şekilde yanıtlıyorsun.

Benchmark & RAG Performance

This model achieves its best performance when combined with a RAG (Retrieval-Augmented Generation) pipeline. Fine-tuning teaches the model domain expertise, terminology, and response format, while RAG provides grounded, factual information from source documents.

Benchmark: 4-Configuration Comparison (10 Questions)

Detailed Question-by-Question Results

(*) Anti-hallucination test questions — these are fictional entities that do not exist.

(**) "No information available" responses on unanswerable questions are counted as correct — honest refusal is preferred over hallucination.

Key findings:

• ARGUS + RAG achieves 10/10 accuracy with zero hallucinations — answers correctly or honestly says "no information available"
• RAG alone improves the base model significantly but still produces hallucinations on edge cases
• ARGUS alone learns domain terminology and format but hallucinates without grounding data
• Base Qwen lacks aviation cybersecurity knowledge entirely (confused Volt Typhoon with League of Legends)

Recommended RAG Setup

• Vector DB: Qdrant
• Embedding Model: intfloat/multilingual-e5-base (Turkish + English)
• LLM Server: llama-server (llama.cpp) with Q5_K_M GGUF

Usage

With Transformers + PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "Qwen/Qwen2.5-14B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto")
model = PeftModel.from_pretrained(model, "yunusshin/argus-qwen25-14b")
tokenizer = AutoTokenizer.from_pretrained(base_model)

messages = [
    {"role": "system", "content": "Sen ARGUS, bir havacılık siber güvenlik uzmanısın."},
    {"role": "user", "content": "EASA Part-IS kapsamında ISMS gereksinimleri nelerdir?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With GGUF (llama-server / Ollama)

A Q5_K_M GGUF quantization (9.8 GB) is also available in this repository.

# llama-server
llama-server --model argus-q5_k_m.gguf --host 0.0.0.0 --port 8080 --ctx-size 4096 --n-gpu-layers 99

# Ollama
ollama create argus -f Modelfile
ollama run argus

Limitations

• Without RAG, the model may hallucinate on topics outside its training data
• Designed specifically for aviation cybersecurity; general cybersecurity knowledge is inherited from the base model
• Regulation article numbers and dates should always be verified against official sources

Training Infrastructure

• Hardware: NVIDIA DGX Spark (GB10 Blackwell), 119.6 GB unified memory
• Framework: Unsloth + TRL (SFTTrainer)

Author

Yunus Şahin

License

Apache 2.0 (following the base model license)