sigmaforge-rule-generator

SigmaForge Rule Generator

Fine-tuned Sigma detection-rule generator based on Qwen/Qwen2.5-Coder-1.5B-Instruct. The model drafts and repairs Sigma YAML from natural-language detections, sample events, or broken rules, then downstream tools compile the YAML to Splunk SPL, Elastic Lucene, and Microsoft Sentinel KQL.

Published by the Aria AI Security Research Team.

Training

The supervised mix covers three dataset configs:

• description_to_sigma — analyst description → Sigma YAML
• sigma_repair — broken rule + validation errors → corrected rule
• logs_to_sigma — positive/negative events → expected rule

Held-out evaluation (n=60, description_to_sigma test)

The generator is reliable at emitting schema-valid, compilable Sigma YAML. Technique tagging is weaker: the model often omits ATT&CK IDs even when the rule body is usable. Pair it with the retrieval + pySigma validation loop in the SigmaForge Space rather than treating raw model output as a deployable rule.

Raw numbers: eval_results.json.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "alirezaaminzadeh/sigmaforge-rule-generator"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")

messages = [
    {
        "role": "system",
        "content": "You are SigmaForge. Write valid Sigma YAML only.",
    },
    {
        "role": "user",
        "content": (
            "Write a Sigma rule for the following detection.\n\n"
            "Detection: Suspicious access to LSASS memory from a non-system process\n"
            "Log source: product=windows, category=process_access\n"
            "MITRE ATT&CK: T1003.001"
        ),
    },
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=700, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Validate every generated rule with pySigma before deploying it to a SIEM.

Related

• Dataset: sigmaforge-detection-rules
• Demo: sigmaforge
• Trainer: sigmaforge-trainer
• Collection: SigmaForge

License

MIT for this fine-tune. Upstream SigmaHQ rules remain under the Detection Rule License (DRL 1.1).