hephaestus-hdfs-0.5b

Hephaestus HDFS Log Anomaly Detector v0.1

A specialized LLM fine-tuned on HDFS (Hadoop Distributed File System) log anomaly detection using QLoRA fine-tuning.

Model Details

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Fine-tuning method: QLoRA (rank=128, alpha=256, dropout=0.05)
• Training: SFT with 8-bit Adam, gradient checkpointing
• Dataset: 33K HDFS log conversations (system/user/assistant format)
• Parameters: 494M (0.5B base + LoRA merged)
• Size: 988 MB (fp16 safetensors)

Performance

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Yusif-v/hephaestus-hdfs-0.5b")
tokenizer = AutoTokenizer.from_pretrained("Yusif-v/hephaestus-0.5b")

prompt = """<|im_start|>system
You are a SOC analyst. Classify as NORMAL or ANOMALY.<|im_end|>
<|im_start|>user
powershell.exe -enc SQBFAFgAIAAoAE4AZQB3AC0ATwBiAGoAZQBjAHQ...<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
# Output: "ANOMALY. Suspicious encoded PowerShell command detected."

Limitations

• 0.5B model — suitable for edge deployment but less accurate than larger variants
• Trained specifically on HDFS logs — may not generalize to other log formats
• English only