inframind-0.5b-grpo

InfraMind: Infrastructure-as-Code Small Language Model

InfraMind is a 0.5B parameter language model fine-tuned for Infrastructure-as-Code (IaC) generation using reinforcement learning (GRPO/DAPO).

Model Description

Why InfraMind?

Unlike traditional fine-tuning (SFT/LoRA) that memorizes patterns, InfraMind uses reinforcement learning with domain-specific rewards to teach the model to reason about infrastructure.

Evaluation Results

Evaluated on InfraMind-Bench (110 held-out test samples) across:

• Terraform (AWS, GCP, Azure)
• Kubernetes (Deployments, Services, Ingress)
• Docker (Dockerfile, docker-compose)
• CI/CD (GitHub Actions, GitLab CI)

Comparison with Other Models

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model
model = AutoModelForCausalLM.from_pretrained("srallabandi0225/inframind-0.5b-grpo")
tokenizer = AutoTokenizer.from_pretrained("srallabandi0225/inframind-0.5b-grpo")

# Generate Terraform
prompt = """### Instruction:
Create Terraform for AWS EC2 instance
### Input:
t3.micro instance type
### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example Output

resource "aws_instance" "web" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.micro"

  tags = {
    Name = "web-server"
  }
}

Supported IaC Categories

Training Details

GRPO (Group Relative Policy Optimization)

First stage training using GRPO:

Training:
  epochs: 3
  batch_size: 16 (effective)
  learning_rate: 5e-6
  beta (KL): 0.04
  generations_per_prompt: 4

LoRA:
  r: 16
  alpha: 32
  target_modules: [q_proj, k_proj, v_proj, o_proj]

DAPO (Direct Advantage Policy Optimization)

Second stage training with DAPO innovations:

Training:
  epochs: 2
  batch_size: 16 (effective)
  learning_rate: 5e-6
  beta (KL): 0.0  # Pure DAPO
  generations_per_prompt: 8

DAPO Innovations:
  1. Clip-Higher: Asymmetric clipping (ε_low=0.2, ε_high=0.28)
  2. Dynamic Sampling: Skip uniform reward batches
  3. Token-Level Loss: Per-token policy gradient
  4. Overlong Punishment: Soft length penalty

Reward Function

Domain-specific reward for IaC quality:

Reward = α × Syntax + β × Correctness + γ × Format

Where:
- Syntax (α=0.4): Valid resource declarations
- Correctness (β=0.3): Correct resource types
- Format (γ=0.3): Proper structure

Hardware Requirements

The 0.5B model is small enough to run on edge devices, making it suitable for:

• Air-gapped environments
• Local development
• CI/CD pipelines
• IoT/Edge infrastructure

Limitations

• IaC-specific: Optimized for infrastructure tasks, not general conversation
• English only: Training data is in English
• No execution: Generates code, does not execute or validate against real infrastructure
• Version-sensitive: Generated code may use older API versions
• Security: Always review generated code for security best practices

Out-of-Scope Uses

• Legal or medical advice
• General-purpose chatbot
• Executing infrastructure changes without human review
• Production deployment without validation

Intended Use

Primary Use Cases

• Generating Terraform configurations
• Creating Kubernetes manifests
• Writing Dockerfiles and docker-compose
• Building CI/CD pipelines
• Infrastructure automation scripting

Users

• DevOps engineers
• Platform engineers
• SREs
• Cloud architects
• Infrastructure developers

Training Data

InfraMind-Bench: 2000+ IaC tasks in Alpaca format

Data format:

{
  "instruction": "Create Terraform for AWS EC2 instance",
  "input": "t3.micro instance type",
  "output": ""
}

Ethical Considerations

• Model may generate insecure configurations if not prompted for security
• Generated infrastructure code should always be reviewed before deployment
• Model does not have access to real infrastructure or credentials
• Users are responsible for validating generated code against their security policies

Citation

@misc{rallabandi2024inframind,
  title={InfraMind: Fine-tuning Small Language Models for Infrastructure-as-Code Generation with Reinforcement Learning},
  author={Rallabandi, Sai Kiran},
  year={2024},
  publisher={HuggingFace},
  url={https://huggingface.co/srallabandi0225/inframind-0.5b-grpo}
}

Links

• GitHub: github.com/saikiranrallabandi/inframind
• GRPO Model: srallabandi0225/inframind-0.5b-grpo
• DAPO Model: srallabandi0225/inframind-0.5b-dapo

Acknowledgments

• Qwen Team for the base model
• DeepSeek for GRPO
• NVIDIA NeMo for DAPO reference
• TRL for training infrastructure

Model Card Contact

Author: Sai Kiran Rallabandi GitHub: @saikiranrallabandi