Valuoty-industry-plc-3B

🧠 PLC Agentic Assistant (IEC 61131-3)

A compact assistant model specialized for industrial PLC programming.
Fine-tuned with Axolotl using QLoRA to follow instructions and generate logic across the five IEC 61131-3 languages: Structured Text (ST), Ladder Diagram ASCII (LD), Function Block Diagram ASCII (FBD), Instruction List (IL), and GRAFCET / SFC.

✨ Highlights

• Domain focus: Industrial automation & PLC engineering
• Skills: Instruction following, code generation, refactoring, SFC design, diagnostics, and troubleshooting
• Languages: ST · LD (ASCII) · FBD (ASCII) · IL · SFC/GRAFCET
• Deployment: Lightweight adapter suitable for edge/on-prem setups

🚀 Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Replace with your merged-weights repo on Hugging Face
model_id = "ICSFR-HF-ORG-01/Valuoty-industry-plc-3B"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16 if torch.cuda.is_available() else "auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "You are a senior PLC engineer. Follow IEC 61131-3 best practices."},
    {"role": "user", "content": "Write a PID function block in Structured Text with anti-windup and usage notes."}
]

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.2,
        top_p=0.9,
        repetition_penalty=1.05,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🗣️ Prompting Cheatsheet (per IEC 61131-3)

Structured Text (ST)

System: You are a PLC expert. Use IEC 61131-3 Structured Text with clear comments.
User: Create FB ConveyorInterlock in ST.
Requirements:
- Inputs: StartCmd, StopCmd, EStop, GuardDoorClosed
- Output: ConveyorRun
- Latch start; drop on EStop or open guard door
- Add diagnostics strings and a test example

Ladder Diagram ASCII (LD)

System: Output Ladder Diagram in clean ASCII. Include rung titles and comments.
User: Build a motor start/stop circuit with seal-in, E-Stop, and overload trip.

Function Block Diagram ASCII (FBD)

System: Output Function Block Diagram in ASCII with clear blocks and signal names.
User: Implement a ramp-up speed profile: TON pre-delay, then PID speed control with setpoint tracking.

Instruction List (IL)

System: Use Instruction List. Comment each instruction briefly.
User: Create IL to compute a moving average over N samples with overflow protection.

GRAFCET / SFC

System: Output SFC (GRAFCET) text with steps S0..Sx, transitions T1..Tx, actions, and safety interlocks.
User: Batch filling: Idle → Fill → Hold → Drain; include E-Stop and LevelHi/Lo interlocks.

🔧 Training Configuration (from Axolotl)

• Base model: Qwen/Qwen2.5-3B
• Adapter: lora (r=24, alpha=32, dropout=0.05)
• Target modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
• Modules to save: ['lm_head']
• Sequence length: 1024
• Quantization (train): 4-bit NF4 (double quant, type=nf4)
• Compute dtype: bfloat16
• Optimizer: adamw_torch, LR 3e-5, weight decay 0.01, max grad norm 1.0
• Schedule: cosine, warmup 0.08
• Batches & epochs: micro-batch 4, grad-accum 8, epochs 3
• Stability: gradient checkpointing True, Flash-Attention True
• Evaluation & logging: strategy steps, eval steps 100, logging 75, load best at end True

Datasets

• Synthetic dataset produce by SPIE ICS.

📊 Performance

<img src="assets/bench.png" width="600" />

Industry PLC MMLU-style benchmark. Our fine-tuned 3B (non-reasoning) PLC model reaches 51.4% accuracy, edging out Mistral Nemo 12B (51.2%) and outperforming larger generalist baselines—Llama 3.1 18B (48.8%) and Mistral Small 24B (48.9%)—while a Qwen2.5 32B (non-reasoning) baseline trails at 42.4%. These results suggest that targeted PLC supervision enables a compact 3B model to match or surpass much larger models, making it practical for on-prem/edge PLC engineering workflows.

🧪 Inference Tips

• Start with temperature=0.2–0.4, top_p=0.9.
• For deterministic scaffolds set temperature=0.0; relax for brainstorming.
• Provide PLC family and safety constraints in the system message to tailor outputs.

⚠️ Limitations & Safety

• Outputs may require adaptation to vendor-specific syntax (Codesys, TwinCAT, Studio 5000, TIA Portal, etc.).
• Ladder/FBD ASCII is illustrative; import into your IDE and verify.
• Not a substitute for standards compliance, FMEA, or SIL/PL validation.
• Always simulate, peer-review, and follow plant safety procedures (IEC 60204-1, ISO 13849).

📄 License

Valuoty AI Model License Agreement 1.0. Review the licenses agreements before commercial use.

📚 Citation

@software{Valuoty-industry-plc-3B,
  title        = {PLC Agentic Assistant (IEC 61131-3) — LoRA on Qwen/Qwen2.5-3B},
  author       = Laurent MAILLE SPIE ICS,
  year         = {2025},
  publisher    = {Hugging Face},
  note         = {Fine-tuned with Axolotl on PLC instruction-following tasks (ST, LD, FBD, IL, SFC).}
}