tiny-guard-4m-en-general-politeness-multiclass-intel

enguard/tiny-guard-4m-en-general-politeness-multiclass-intel

This model is a fine-tuned Model2Vec classifier based on minishlab/potion-base-4m for the general-politeness-multiclass found in the Intel/polite-guard dataset.

Installation

pip install model2vec[inference]

Usage

from model2vec.inference import StaticModelPipeline

model = StaticModelPipeline.from_pretrained(
  "enguard/tiny-guard-4m-en-general-politeness-multiclass-intel"
)


# Supports single texts. Format input as a single text:
text = "Example sentence"

model.predict([text])
model.predict_proba([text])

Why should you use these models?

• Optimized for precision to reduce false positives.
• Extremely fast inference: up to x500 faster than SetFit.

This model variant

Below is a quick overview of the model variant and core metrics.

Confusion Matrix

<details> <summary><b>Full metrics (JSON)</b></summary>

{
  "impolite": {
    "precision": 0.9896124650419497,
    "recall": 0.9782780410742496,
    "f1-score": 0.9839126117179742,
    "support": 2532.0
  },
  "neutral": {
    "precision": 0.8709677419354839,
    "recall": 0.8989424206815512,
    "f1-score": 0.8847340015420201,
    "support": 2553.0
  },
  "polite": {
    "precision": 0.8721425803951957,
    "recall": 0.876899104012466,
    "f1-score": 0.8745143745143745,
    "support": 2567.0
  },
  "somewhat polite": {
    "precision": 0.8363563079403467,
    "recall": 0.8143642072213501,
    "f1-score": 0.8252137601908929,
    "support": 2548.0
  },
  "accuracy": 0.8919607843137255,
  "macro avg": {
    "precision": 0.8922697738282439,
    "recall": 0.8921209432474042,
    "f1-score": 0.8920936869913154,
    "support": 10200.0
  },
  "weighted avg": {
    "precision": 0.8920691454072528,
    "recall": 0.8919607843137255,
    "f1-score": 0.8919133038383807,
    "support": 10200.0
  }
}

</details> <details> <summary><b>Sample Predictions</b></summary> </details> <details> <summary><b>Prediction Speed Benchmarks</b></summary> </details>

Other model variants

Below is a general overview of the best-performing models for each dataset variant.

Resources

• Awesome AI Guardrails: https://github.com/enguard-ai/awesome-ai-guardails
• Model2Vec: https://github.com/MinishLab/model2vec
• Docs: https://minish.ai/packages/model2vec/introduction

Citation

If you use this model, please cite Model2Vec:

@software{minishlab2024model2vec,
  author       = {Stephan Tulkens and {van Dongen}, Thomas},
  title        = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year         = {2024},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.17270888},
  url          = {https://github.com/MinishLab/model2vec},
  license      = {MIT}
}