gpt-oss-20b-cve-cybersecurity

GPT-OSS-20B CVE Cybersecurity Model

Model Description

This model is a fine-tuned version of GPT-OSS-20B specialized for CVE (Common Vulnerabilities and Exposures) analysis and security recommendation generation using DoRA (Weight-Decomposed Low-Rank Adaptation).

Key Features

• Analyzes CVE vulnerabilities and provides actionable security recommendations
• Trained on 5,000+ diverse CVE policy recommendations
• Uses DoRA for efficient fine-tuning
• Provides structured recommendations with rationale
• Identifies security risks and suggests mitigation strategies

Quick Start

Installation

pip install transformers torch

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "sainikhiljuluri/gpt-oss-20b-cve-cybersecurity",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("sainikhiljuluri/gpt-oss-20b-cve-cybersecurity")

# Prepare CVE analysis request
messages = [
    {
        "role": "system",
        "content": "You are an expert cybersecurity analyst specializing in vulnerability assessment and remediation. Analyze CVE information and provide actionable security recommendations with clear rationale."
    },
    {
        "role": "user",
        "content": """Analyze the following vulnerability and provide security recommendations:

CVE ID: CVE-2024-1234
Vulnerability Summary: SQL injection vulnerability in web application allows remote attackers to execute arbitrary SQL commands
CVSS Score: 9.8
Weakness Type: SQL Injection
CWE Code: CWE-89"""
    }
]

# Generate response
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(
    outputs[0][inputs.shape[-1]:],
    skip_special_tokens=True
)

print(response)

Training Details

Dataset

• Source: CVE Policy Recommendations (5,000+ diverse examples)
• Format: Structured CVE information with expert recommendations
• Split: 98% training, 2% evaluation

Training Configuration

• Method: DoRA (Weight-Decomposed Low-Rank Adaptation)
• LoRA Rank: 32
• LoRA Alpha: 32
• Dropout: 0.10
• Base Precision: BF16
• Merged Precision: FP16
• Epochs: 3
• Batch Size: 2 (effective: 16 with gradient accumulation)
• Learning Rate: 1.5e-4
• Training Platform: Google Colab A100 GPU

Training Results

• Final Perplexity: 1.57 (Excellent)
• Average Loss: 0.45
• BLEU-4 Score: 0.4954
• ROUGE-L F1: Not yet calculated
• Semantic Similarity: 0.6305
• Quality Retention: 94.3%

Capabilities

CVE Analysis

The model can analyze various types of vulnerabilities including:

• SQL Injection
• Cross-Site Scripting (XSS)
• Authentication Bypass
• Denial of Service (DoS)
• Buffer Overflow
• Remote Code Execution
• And many more...

Recommendation Format

The model provides structured recommendations including:

• Recommended Action: Specific steps to mitigate the vulnerability
• Rationale: Explanation of why these actions are necessary
• Risk Assessment: Evaluation of severity and impact
• Implementation Details: Technical guidance for remediation

Example Output

Input:

CVE ID: CVE-2024-SQL-001
Vulnerability Summary: SQL injection in authentication module
CVSS Score: 9.8
Weakness Type: SQL Injection

Output:

Recommended Action:
1. Immediately implement parameterized queries/prepared statements
2. Update web application framework to latest patched version
3. Enable Web Application Firewall (WAF) rules
4. Conduct comprehensive code review of authentication module
5. Deploy input validation and sanitization

Rationale:
Critical CVSS score of 9.8 indicates immediate action required. SQL injection
in authentication module poses severe risk of unauthorized access and data breach.
Parameterized queries prevent SQL injection at the source by separating SQL logic
from user input...

Performance Metrics

Model Architecture

• Base Model: GPT-OSS-20B
• Parameters: ~20 billion
• Precision: FP16 (merged model)
• Adapter Type: DoRA (rank-32)
• Context Length: 2048 tokens
• Model Size: ~40GB

Deployment

HuggingFace Inference Endpoints

import requests

API_URL = "https://YOUR-ENDPOINT.endpoints.huggingface.cloud"
headers = {"Authorization": f"Bearer {YOUR_HF_TOKEN}"}

def analyze_cve(cve_info):
    payload = {
        "inputs": cve_info,
        "parameters": {
            "max_new_tokens": 512,
            "temperature": 0.7,
            "top_p": 0.9
        }
    }
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

Local Deployment

# Load model locally
model = AutoModelForCausalLM.from_pretrained(
    "sainikhiljuluri/gpt-oss-20b-cve-cybersecurity",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

Limitations

1. Domain-Specific: Optimized for CVE analysis, may not generalize well to other tasks
2. Training Data: Limited to vulnerabilities seen in training data
3. No Real-Time Data: Model knowledge is based on training data cutoff
4. Requires Context: Best results when provided with complete CVE information
5. Not a Replacement: Should complement, not replace, professional security analysis

Ethical Considerations

• This model is for research and educational purposes
• Always validate security recommendations with professional tools
• Do not rely solely on AI for critical security decisions
• Use responsibly and ethically in cybersecurity contexts

Citation

If you use this model in your research, please cite:

@misc{gpt-oss-20b-cve-2025,
  author = {Sainikhil Juluri},
  title = {GPT-OSS-20B CVE Cybersecurity Model},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/sainikhiljuluri/gpt-oss-20b-cve-cybersecurity}
}

Training Details

Training Date: 2025-11-04

Hardware: Google Colab A100 GPU

Framework:

• PyTorch 2.8.0+cu126
• Transformers
• PEFT (DoRA)

Contact

For questions or collaborations:

• Open an issue on the model repository
• Connect via HuggingFace discussions

License

Apache 2.0 License

Acknowledgments

• Base model: GPT-OSS-20B by OpenAI
• Training method: DoRA (Weight-Decomposed Low-Rank Adaptation)
• Dataset: CVE Policy Recommendations

---

Built with: Transformers • PEFT • DoRA • PyTorch