governance-risk-classifier

Governance Risk Classifier

A small experimental baseline for classifying synthetic AI governance scenarios into coarse engineering risk tiers.

The model is part of the AIGovDev public AI governance engineering portfolio.

Model architecture

The pipeline uses:

• categorical feature preprocessing
• one-hot encoding
• logistic regression
• balanced class weights

Input features:

• sector
• impact
• decision autonomy
• human oversight
• post-deployment monitoring
• traceability
• technical documentation

Target labels

The original governance labels are mapped into three coarse tiers:

• low and limited → lower
• high → high
• unacceptable → unacceptable

These labels are engineering annotations only.

They are not legal classifications and must not be interpreted as determining compliance with the EU AI Act or any other regulation.

Training data

The model is trained on:

aigovdev/ai-governance-scenarios

The current dataset contains 12 synthetic scenarios:

• 6 lower
• 3 high
• 3 unacceptable

The scenarios were manually constructed for experimentation and demonstration.

They are not production data.

Evaluation

The current prototype uses stratified 3-fold cross-validation.

Results:

Confusion matrix:

lower         6  0  0
high          0  3  0
unacceptable  0  0  3

These results must be interpreted with extreme caution.

The dataset contains only 12 synthetic, manually constructed examples. The perfect cross-validation result is therefore primarily evidence that the current feature patterns are easily separable within this toy dataset.

It is not evidence of real-world generalization, production readiness, regulatory validity, or calibrated governance-risk estimation.

Intended use

This model is suitable for:

• educational experiments
• AI governance pipeline prototyping
• demonstrating transparent tabular ML workflows
• testing governance-risk feature engineering
• portfolio and research-oriented demonstrations

Not intended for

The model must not be used as:

• a legal compliance classifier
• an EU AI Act classification system
• a production governance decision engine
• a substitute for legal or technical risk assessment
• a validated risk-scoring system

Reproducibility

The repository contains:

• src/train.py
• requirements.txt
• model.joblib
• metrics.json

The training script downloads the public dataset directly from Hugging Face and reproduces the model pipeline and evaluation.

Limitations

Major limitations include:

• extremely small dataset
• synthetic scenarios
• manually constructed labels
• simplified governance dimensions
• no external validation set
• no calibration analysis
• no real deployment data

Future work

Possible extensions include:

• larger governance scenario datasets
• independently annotated examples
• real-world control evidence
• lifecycle and incident features
• probability calibration
• external validation
• benchmark comparison
• legal-risk separation from engineering-risk signals

AIGov

AIGov builds infrastructure for transparent, auditable, and accountable AI systems.

Website: https://govbase.dev

License

Apache-2.0

Inference

The repository includes inference.py for local predictions.

Example:

python inference.py --json '{
  "sector": "financial_services",
  "impact": "high",
  "decision_autonomy": "automated",
  "human_oversight": "none",
  "monitoring": "none",
  "traceability": "none",
  "technical_documentation": "partial"
}'

Example output:

{
  "risk_tier": "unacceptable",
  "class_probabilities": {
    "high": 0.24612,
    "lower": 0.08766,
    "unacceptable": 0.66622
  }
}

The returned class probabilities are model scores from the logistic-regression baseline. They should not be interpreted as calibrated probabilities of legal or real-world governance risk.