tinyllama-cyber-finetuned

TinyLlama-1.1B Cybersecurity Fine-Tuned

A fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 trained on cybersecurity instruction-response data using LoRA (Low-Rank Adaptation).

Model Details

Training Details

How to Load

With Unsloth (recommended):

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = "Aaqisher667/tinyllama-cyber-finetuned",
    max_seq_length = 512,
    dtype          = None,
    load_in_4bit   = True,
)
FastLanguageModel.for_inference(model)

Example usage:

messages = [
    {"role": "system", "content": "You are a cybersecurity expert assistant."},
    {"role": "user",   "content": "What is a SQL injection attack?"},
]
chat = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
inputs = tokenizer(chat, return_tensors="pt").to(model.device)
import torch
with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens = 256,
        temperature    = 0.7,
        do_sample      = True,
    )
response = tokenizer.decode(
    out[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)
print(response)

Intended Use

This model is intended for:

• Cybersecurity question answering
• Security concept explanation
• Vulnerability analysis assistance
• On-device / edge deployment scenarios requiring a lightweight cybersecurity AI assistant

Limitations

• Small model size (1.1B) — responses may lack depth compared to larger models
• Fine-tuned on a limited dataset (~2,500 examples)
• Not suitable for generating exploit code or offensive security tooling
• Always verify cybersecurity advice from authoritative sources

Hardware Requirements

Runs comfortably on a free Google Colab T4 GPU.

Training Data

This model was fine-tuned on AlicanKiraz0/Cybersecurity-Dataset-v1.