howzer-risk-scorer

HowzerRiskScorer — Customer Feedback Risk Assessment

A lightweight MLP model (80K parameters) that scores customer feedback risk across multiple dimensions. Designed for German-language customer service contexts (e.g., automobile clubs, insurance).

Model Description

HowzerRiskScorer replaces a 2,000-line rule-based risk engine with a learned model that takes pre-computed NLP pipeline features and outputs multi-dimensional risk scores.

Architecture: MLP with BatchNorm + GELU + Dropout (256 → 128 → 64) with multi-head outputs.

Input Features (120 dimensions)

The model expects a pre-computed feature vector from an upstream NLP pipeline:

Outputs

Tier thresholds: critical ≥ 0.88, high ≥ 0.72, medium ≥ 0.38, low < 0.38

Performance

Evaluated on a held-out test set (437 samples) of German customer feedback:

Per-tier F1

Usage

ONNX Runtime (recommended)

import numpy as np
import onnxruntime as ort

session = ort.InferenceSession("howzer_risk_scorer.onnx")
features = np.random.randn(1, 120).astype(np.float32)  # your feature vector

outputs = session.run(None, {"features": features})
escalation, churn, brand, revenue, global_risk, tier_logits, confidence = outputs

tier_names = ["low", "medium", "high", "critical"]
predicted_tier = tier_names[tier_logits[0].argmax()]
print(f"Risk: {global_risk[0,0]:.3f}, Tier: {predicted_tier}")

Python Inference Helper

from inference import RiskScorer

scorer = RiskScorer.from_onnx("howzer_risk_scorer.onnx")
result = scorer.predict(features)  # returns RiskAssessment dataclass

print(f"Global risk: {result.global_risk:.3f}")
print(f"Tier: {result.tier} (confidence: {result.confidence:.3f})")
print(f"Components: esc={result.escalation:.3f}, churn={result.churn:.3f}")

PyTorch

import torch
from src.model.risk_scorer import HowzerRiskScorer

model = HowzerRiskScorer(input_dim=120, hidden_dims=(256, 128, 64))
model.load_state_dict(torch.load("model.safetensors"))
model.eval()

with torch.no_grad():
    preds = model(torch.randn(1, 120))
    print(f"Risk: {preds['global_risk'].item():.3f}")

Training Details

• Data: 4,351 unique German customer feedback samples (4,000 from benchmark corpus + 351 from curated golden set)
• Silver labels: Generated by rule-based risk engine v6.5.2
• Oversampling: sqrt strategy for class imbalance (critical tier: 1.7% of data)
• Loss: Focal loss (γ=1.5) + MSE score regression + BCE confidence calibration
• Optimizer: AdamW (lr=5e-4, weight_decay=1e-2)
• Schedule: Cosine annealing with linear warmup
• Early stopping: Patience 25, best checkpoint by validation loss
• Hardware: AMD iGPU via DirectML

Benchmark: MLP vs LLMs

We compared the 80K-parameter MLP against general-purpose LLMs on the same risk scoring task. LLMs received the raw German feedback text and were prompted to produce structured risk assessments.

Key findings:

• The domain-specific MLP achieves 10x higher accuracy than 7-8B parameter LLMs
• LLMs are 27,000x slower per inference
• LLMs interpret "risk" differently than the calibrated rule engine, resulting in scores far from ground truth (MAE ~0.6 vs 0.08)
• This demonstrates the value of domain-specific fine-tuning over general-purpose LLMs for structured prediction tasks

Benchmark on 50 German customer feedback samples. LLMs were prompted with text only; MLP used pre-computed pipeline features.

Limitations

• Critical tier performance is limited (F1=0.25) due to data scarcity — only 74 critical samples in the entire dataset
• Requires pre-computed NLP features (sentiment, emotion, root cause) — cannot process raw text directly
• Trained on ADAC (German automobile club) customer feedback — domain transfer to other contexts not validated
• Silver labels come from rule engine, not human annotators

Files

Citation

@misc{howzer-risk-scorer-2026,
  title={HowzerRiskScorer: Learned Customer Feedback Risk Assessment},
  year={2026},
  publisher={Hugging Face},
}