eu-ai-recruiter-fit-classifier

EU-AI-Act-Aware Resume-Job Fit Classifier

A two-stage resume-to-job-description fit classifier, built with the EU AI Act's transparency and bias-disclosure requirements in mind. Recruitment scoring/ranking tools are explicitly treated as high-risk AI systems under the Act's draft guidance, so this model prioritizes honest disclosure of its limitations over polished-looking performance numbers.

Two-stage design

• Stage 1 (reliable): No Fit vs Some Fit — 66% accuracy (vs. 51% majority baseline), balanced across both classes.
• Stage 2 (weak, use with caution): Good Fit vs Potential Fit — only 57% accuracy (vs. 51% baseline) when applied to "Some Fit" pairs. This distinction should be treated as a soft hint, never a standalone decision.

This split exists because a single 3-class model conflated a distinction the data does not clearly support (Good vs Potential Fit) with one it does (Fit vs No Fit). Forcing false confidence on the former would have been misleading.

What this model outputs

For each resume/job description pair:

• A primary Fit / No Fit prediction with confidence
• The resume-job semantic similarity score
• The top words/features driving the decision (explainability)
• A secondary Good/Potential signal, explicitly flagged as low-reliability

Known limitations (disclosed deliberately)

1. Truncation was not the fix — reframing was. An earlier transformer-based approach (DistilBERT) performed at or below baseline (47%) because resumes average 1,207 tokens, far exceeding the 512-token limit, silently cutting off later resume sections (often the Skills section). A TF-IDF + similarity-feature approach was used instead, since it is not token-limited.
2. Spurious correlations were found and partially fixed. An earlier version of this model relied partly on company/organization names (e.g. specific employers) as predictive features. Named-entity stripping (spaCy NER) removed most of this, with accuracy dropping only 1 point (67% -> 66%), confirming the model's real signal was not primarily coming from these artifacts.
3. A residual bias-shaped pattern remains unresolved. Certain legitimate skill words (e.g. "electrical", "engineering", "data", "analytics") still correlate with "No Fit" predictions. This may reflect genuine class imbalance in which job categories are underrepresented among "Fit" labels in the training data, rather than a universal truth about those skills. This has NOT been fully root-caused and should be investigated further before any production use.
4. This model should not make autonomous hiring decisions. Consistent with the EU AI Act's human-oversight requirement for high-risk employment AI systems, all outputs should be reviewed by a human recruiter, and no candidate should be rejected solely on this model's output.

Intended use

Portfolio / research demonstration of building an explainable, limitation-aware resume-screening classifier. Not validated for production hiring decisions.

Training data

cnamuangtoun/resume-job-description-fit — 6,241 train / 1,759 test resume-job pairs, labeled No Fit / Potential Fit / Good Fit.

Author

Syed Sibte Hassan Shah - International Recruiter & AI Automation Specialist, applying EU AI Act / GDPR compliance background to practical NLP model-building.