permitguard-risk-classifier

PermitGuard — PTW Risk Classifier (TF-IDF + Logistic Regression + Keyword Prior)

Text-only bilingual (FA/EN) risk-level and work-type classifiers for synthetic Permit-to-Work narratives. Ground-truth risk comes from 13 deterministic, auditable rules on structured fields; the model sees only free text, so it is scored on whether it can recover that rule-derived label (including the designed ~19% text/flag mismatches). Part of Aria AI Aria SafeOps → Control of Work.

Data honesty

Trained only on permitguard-ptw-samples — 100% synthetic, no real permits. Do not treat the numbers below as field PTW performance.

Held-out results (125 permits)

Work-type accuracy is 100% because the synthetic narratives name the work type in the first sentence (by design, for a readable demo). It is not a claim that work-type NER is solved in the field. Critical recall is the headline safety metric; the two misses are consistent with the designed text/flag mismatch slice.

SIMOPS is a deterministic rule engine, not this sklearn model. 260 pairs, 50% conflict, five rules (SIMOPS-01…05) each fired 26 times. Precision/recall vs those labels would be circular 100% and is therefore not reported as a learned KPI.

Files

• tfidf_vectorizer.joblib — word (1–2gram) + char_wb (3–5gram) TF-IDF
• risk_classifier.joblib — multinomial Logistic Regression (balanced)
• worktype_classifier.joblib — same features → 7 work types
• eval_results.json / benchmark_report.json / manifest.json

Inference also applies a small keyword prior (ContractGuard-style): if the text explicitly says a critical precaution was not done, critical probability is boosted. See src/permitguard/models.py.

Related

• Dataset: alirezaaminzadeh/permitguard-ptw-samples
• Space: alirezaaminzadeh/permitguard-ptw-risk-classifier
• Sibling RAG: alirezaaminzadeh/petrosafe-rag-fa
• Product: aria-ai.ir

MIT · Aria AI Engineering Team