equibert-inclusive-language

EquiBERT — Inclusive Language Scorer

Model ID: SallySims/equibert-inclusive-language

Dual-output model for inclusive language analysis:

1. Token-level BIO tagging — identifies and spans problematic language
2. Document-level score — 0.0 (highly exclusive) to 1.0 (highly inclusive)

Span Tag Labels (13 classes)

O, B-GENDERED, I-GENDERED, B-ABLEIST, I-ABLEIST, B-RACIAL_CODE, I-RACIAL_CODE, B-AGE_CODED, I-AGE_CODED, B-EXCLUSIONARY, I-EXCLUSIONARY, B-AGGRESSIVE, I-AGGRESSIVE

Score Interpretation

Usage

text = "We need a rock star developer who can dominate the roadmap."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
# outputs      = model(**inputs)
# span_tags    = outputs.logits.argmax(-1)       # token-level BIO tags
# incl_score   = outputs.hidden_states[0]         # 0-1 document score

Model Description

EquiBERT is a multi-task DEI (Diversity, Equity and Inclusion) transformer built on a dual-encoder backbone that fuses RoBERTa-base and DeBERTa-v3-base via a learned weighted sum (α parameter). The fused representation is fed into task-specific heads covering 17 distinct DEI analysis tasks.

Organisation: SallySims Framework: PyTorch + HuggingFace Transformers Backbone: RoBERTa-base + DeBERTa-v3-base (dual encoder, fused) Language: English Domain: Organisational DEI text — HR communications, policies, job descriptions, performance reviews, leadership statements, reports

Architecture

Input Text
    │
    ├──▶ RoBERTa-base encoder ──▶ Linear projection
    │                                     │
    └──▶ DeBERTa-v3-base encoder ──▶ Linear projection
                                          │
                              Weighted fusion (learned α)
                                          │
                                   Layer Norm + Dropout
                                          │
                              Task-specific head (see below)

Training Data

Trained on synthetic DEI organisational text generated by the EquiBERT synthetic data pipeline, covering 20 DEI categories across HR, policy, leadership, and workforce analytics domains. For production use, fine-tune on real labelled DEI data.

Limitations

• Trained on synthetic data — predictions should be validated before use in real HR or policy decisions.
• English-only.
• Not a substitute for qualified DEI practitioners or legal advice.
• May reflect biases present in the training corpus.

Citation

If you use EquiBERT in your research, please cite:

@misc{equibert2024,
  author    = {SallySims},
  title     = {EquiBERT: A Multi-Task DEI Transformer},
  year      = {2024},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/SallySims}
}

All EquiBERT Models