chaosops-grpo-lora-p2

ChaosOps AI — GRPO LoRA Adapter

LoRA adapter for Qwen 2.5-1.5B-Instruct, fine-tuned with GRPO (Group Relative Policy Optimization, via TRL) on the ChaosOps AI multi-agent incident-response environment.

What ChaosOps trains

Four LLM agents — SRE · Developer · Manager · Oversight — handle production-incident scenarios (DB deadlock, memory leak, bad config push, autoscaler cost-cut by a rogue AI, misrouted traffic by a rogue load-balancer, cascade, DNS outage, disk full, rogue deploy bot) under partial observability. The Oversight agent is rewarded for catching when another AI in the fleet caused the incident before the team applies a fix.

Training recipe

The reward is composed from four named OpenEnv-style rubrics: resolution, mttr, oversight, cascade — see the ChaosOps source.

Files

• adapter_model.safetensors + adapter_config.json — the LoRA itself
• training_metrics.json — per-log reward + loss + KL stream
• learning_curve.png — reward curve (axis-labelled, 150 dpi)

How to use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct", device_map="auto"
)
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "helloAK96/chaosops-grpo-lora")

Or activate it directly inside the live ChaosOps Space by setting the Space secret CHAOSOPS_ADAPTER_PATH=helloAK96/chaosops-grpo-lora — the Space will lazily snapshot-download the adapter on first request and route the trained policy through it.

Results

Evaluation across 5 seeds × 9 failure types × 3 tiers (540 episodes per policy):

Mean cumulative episode reward / resolution rate. Trained beats random by +94.6 (EASY) / +345.4 (MEDIUM) / +522.4 (HARD) points and lifts MEDIUM-tier resolution rate to 40% (vs the untrained Qwen's 33%). The training-time reward curve in learning_curve.png shows HARD-tier mean reward (−4.43) ending up better than EASY-tier mean (−6.11) — the curriculum let the model absorb easier-tier dynamics before being pushed into HARD.

Links

• 🚀 Live demo (HF Space): https://huggingface.co/spaces/helloAK96/chaosops
• 📄 Source repo: https://github.com/vatsalllll/chaos_ops
• 📒 Training notebook: notebooks/colab_train.ipynb
• 🛠️ Reward rubric system: chaosops/rewards/reward_fn.py

Citation

@misc{chaosops_ai_2026,
  title  = {ChaosOps AI: a multi-agent incident-response gym with rogue-agent detection},
  author = {ChaosOps AI Team},
  year   = {2026},
  url    = {https://huggingface.co/spaces/helloAK96/chaosops}
}