Tini-Cybersec-8B-A1B

Tini-Cybersec-8B-A1B 🛡️🧠

<p align="center"> <img src="thumbnail.jpeg" width="1000" alt="Tini1.5-8B-A1B Logo"> </p>

Tini-Cybersec-8B-A1B is a specialized fine-tuned model based on the LiquidAI/LFM2.5-8B-A1B architecture. It is customized to perform complex Cybersecurity tasks, security analysis, threat modeling, and vulnerability assessment, while preserving and enhancing reasoning and Chain-of-Thought (CoT) capabilities.

The model is SFT-trained using a carefully curated dataset mix of 185,002 records comprising both deep security knowledge and structured step-by-step reasoning paths.

---

📊 Dataset & Matrix Distribution

The SFT training data is a balanced mixture of domain-specific cybersecurity instruction datasets and general reasoning datasets (CoT), filtered to remove empty/zero-token records.

1. Dataset Components

2. Domain Composition

• Cybersecurity Core: 94.23% (~174,328 records)
• Pure Reasoning & Chain-of-Thought (CoT): 5.77% (~10,674 records)

---

📈 Dataset Token Statistics

Calculated using the LiquidAI/LFM2.5-8B-A1B tokenizer:

• Total Records: 185,002
• Total Tokens: 159,858,904 tokens
• Average Token Length: 864.09 tokens per record
• Min Token Length: 54 tokens
• Max Token Length: 78,313 tokens

Token length distribution:

  < 1,000 tokens        : 162,036 records (87.59%) █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █
  1,000 - 10,000        : 22,632 records (12.23%)  █ █ █
  10,000 - 20,000       : 182 records (0.10%)
  20,000 - 30,000       : 77 records (0.04%)
  30,000 - 50,000       : 63 records (0.03%)
  50,000 - 100,000      : 12 records (0.01%)

---

🏆 Evaluation Results (CS-Eval Benchmark)

Tini-Cybersec-8B-A1B has been evaluated on the CS-Eval Benchmark (a comprehensive cybersecurity evaluation benchmark for Large Language Models) and is published on the CS-Eval Leaderboard (under submission name DungNVT-ISELAB).

The model achieved a Comprehensive Score of 76.65%, demonstrating robust capabilities across all domains of system, network, and application security:

---

⚙️ Training Hyperparameters (SFT)

The model was SFT-trained using Unsloth and Hugging Face Trainer with sequence packing to optimize throughput:

---

💬 Prompt Format & Templates

This model follows the ChatML format and supports nested <think> tags for reasoning models.

Template Structure:

<|im_start|>system
You are a helpful and knowledgeable cybersecurity expert assistant. You answer all user queries step by step with reasoning.<|im_end|>
<|im_start|>user
[Your cybersecurity query / task here]<|im_end|>
<|im_start|>assistant
<think>
[Step-by-step thinking process / Chain-of-Thought (CoT)]
</think>
[Detailed response / action plan / explanation]
<|im_end|>

---

🚀 How to Load and Use

To load this model with Hugging Face's transformers library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "./Tini-Cybersec-8B-A1B_26062026"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Inference example
messages = [
    {"role": "system", "content": "You are a cybersecurity expert assistant."},
    {"role": "user", "content": "What is SQL Injection? And how to prevent it?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.6,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

---

📄 License & Attribution

• Base Model: Licensed under the Apache-2.0 license by LiquidAI.
• Fine-tuned Weights: Apache-2.0 License.
• Dataset Attribution: Please credit the original authors of AlicanKiraz0/Cybersecurity-Dataset-Heimdall-v1.1, AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1, Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset, nohurry/Opus-4.6-Reasoning-3000x-filtered, Jackrong/DeepSeek-V4-Distill-8000x, and Jackrong/Qwen3.5-reasoning-700x.