MITRE-v15-tactic-bert-case-based

MITRE-v15-tactic-bert-case-based

It's a fine-tuned model from mitre-bert-base-cased on the MITRE ATT&CK version 15 procedure dataset. It achieves

• loss:0.057
• accuracy:0.87

on evaluation dataset.

Intended uses & limitations

You can use the fine-tuned model for text classification. It aims to identify the tactic that the sentence belongs to in MITRE ATT&CK framework. A sentence or an attack may fall into several tactics.

Note that this model is primarily fine-tuned on text classification for cybersecurity. It may not perform well if the sentence is not related to attacks.

How to use

You can use the model with Tensorflow.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_id = "sarahwei/MITRE-tactic-bert-case-based"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    # device_map="auto",
)
question = 'An attacker performs a SQL injection.'
input_ids = tokenizer(question,return_tensors="pt")
outputs = model(**input_ids)
logits = outputs.logits
sigmoid = torch.nn.Sigmoid()
probs = sigmoid(logits.squeeze().cpu())
predictions = np.zeros(probs.shape)
predictions[np.where(probs >= 0.5)] = 1
predicted_labels = [model.config.id2label[idx] for idx, label in enumerate(predictions) if label == 1.0]

Training procedure

Training parameter

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 0
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• warmup_ratio: 0.01
• weight_decay: 0.001

Training results