mistralv0.3-cybersec

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

This model is a fine-tuned version of mistralai/Mistral-7B-v0.3, adapted specifically for cybersecurity-related tasks using the AlicanKiraz0/Cybersecurity-Dataset-v1. The model has been further trained using supervised fine-tuning with LoRA (Low-Rank Adaptation) to enhance its ability to answer questions and generate content relevant to information security, threat analysis, incident response, application security, and other cybersecurity domains.

The training objective was to make the base model more effective for real-world cybersecurity use cases, including both offensive and defensive security topics. The fine-tuned model is suitable for tasks such as question answering, incident response simulation, CVE summarization, and security education.

Base Model: mistralai/Mistral-7B-v0.3 Fine-Tuning Approach: Parameter-efficient fine-tuning with LoRA (PEFT) Domain: Cybersecurity (offensive & defensive, information security, vulnerability analysis, incident response, etc.) Data: Publicly available, expert-curated cybersecurity texts and structured records Intended Use: This model is designed to assist cybersecurity professionals, researchers, and educators with high-quality responses and reasoning in the cybersecurity domain. It can be used for chatbots, research assistants, automated knowledge extraction, and educational tools.

Limitations:

The model may generate outdated or incorrect information if the data is out of date. Should not be relied upon for critical, real-world incident response without human oversight. Not suitable for generating or promoting illegal or unethical hacking activities.

Model type / Language(s) / License / Finetuned from

Model type: Causal Language Model (Decoder-only, LoRA fine-tuned) Language(s): English (en) License: MIT Finetuned from: mistralai/Mistral-7B-v0.3

Direct Use

The model can be used as a conversational assistant or question-answering system for cybersecurity-related topics. Intended users are cybersecurity professionals, students, and researchers.

Downstream Use

Can be integrated into educational tools, automated incident response simulators, or red-team/blue-team training assistants.

Out-of-Scope Use

Should not be used for automating actual attack scenarios, generating exploit code, or in critical security systems without human oversight.

Bias, Risks, and Limitations

Model may reflect biases present in public cybersecurity datasets. It may hallucinate or return outdated information and is not a substitute for professional judgment.

How to Get Started

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.3", device_map="auto", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.3")
lora_model = PeftModel.from_pretrained(base_model, "[your-hf-username]/[your-model-name]")

prompt = "User: What is a buffer overflow?\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = lora_model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data

See AlicanKiraz0/Cybersecurity-Dataset-v1 for details and license.

Training Procedure & Hyperparameters

Precision: fp16 mixed precision Batch size: 24 Epochs: 52 Learning rate: 2e-4 LoRA rank (r): 8 LoRA alpha: 16

Evaluation

Evaluated on a held-out subset of the same dataset using cross-entropy loss (final value: ~0.81). Human inspection suggests strong security-domain alignment, but thorough downstream task evaluation is ongoing.

Environmental Impact

Hardware Type: NVIDIA A100 Hours used: 2 Cloud Provider: Google Colab

Technical Specifications

Architecture: Mistral-7B, decoder-only transformer Fine-tuning library: PEFT 0.15.2 (LoRA) Software: transformers, peft, bitsandbytes, datasets

Model Card Authors [optional]

eademir

Model Card Contact

https://huggingface.co/eademir

Citation

If you use this model, please cite it as:

@misc{eademir_mistral_cybersec_2024,
  title        = {Mistral-7B-v0.3 Fine-tuned on Cybersecurity-Dataset-v1},
  author       = {Eray Aydemir},
  howpublished = {\url{https://huggingface.co/eademir/mistralv0.3-cybersec}},
  year         = {2024},
  note         = {Fine-tuned Mistral-7B-v0.3 on AlicanKiraz0/Cybersecurity-Dataset-v1}
}

### Framework versions

- PEFT 0.15.2