Downloads · 30 days
6
8% of all-time downloads
14.0.1-ms-marco-MiniLM-L6-v2
acuvity/14.0.1-ms-marco-MiniLM-L6-v2
14.0.1-ms-marco-MiniLM-L6-v2 is a text classification model from acuvity. Use it when you need a label for a piece of text. It is set up for sentence-transformers.
acuvity/intent-action is a MiniLM-L6 Cross-Encoder fine-tuned for tool-use alignment detection - classifying whether an AI agent's action is aligned with its given task.
All-time downloads
72
Public
Parameters
22.7M
90.9 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors90.9 MB · 99%
At a glance
- Task
- Text Classification
- Library
- sentence-transformers
- Model type
- bert
- Access
- Public
- Created
- Jun 30, 2026
- Updated
- Jun 30, 2026
- SHA
- 77db7eba
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
acuvity/intent-action
Model Description
acuvity/intent-action is a MiniLM-L6 Cross-Encoder fine-tuned for tool-use alignment detection -
classifying whether an AI agent's action is aligned with its given task.
- Architecture:
cross-encoder/ms-marco-MiniLM-L6-v2(22M parameters) - Task: Binary classification - aligned (0) vs misaligned (1)
- Training method: Supervised fine-tuning with Binary Cross-Entropy loss
- Loss: BCE
Training Hyperparameters
| Parameter | Value |
|---|---|
| Dataset | acuvity/tool-use-alignment |
| Epochs | 5 |
| Batch size | 32 |
| Learning rate | 1.00e-05 |
| Weight decay | 0.01 |
| Warmup ratio | 0.1 |
| LLRD | No |
| Early stopping patience | 3 |
| Eval every (steps) | 1000 |
Test Results
| Metric | Value |
|---|---|
| F1 | 0.9917 |
| AUPR | 0.9989 |
| Precision | 0.994 |
| Recall | 0.9895 |
| TP / TN / FP / FN | 8,394 / 8,398 / 71 / 75 (n=16,938) |
| Threshold (τ) | 0.8492 |
| Temperature (T) | 1 |
Training Curves
Calibration & Safety Gating
Each score maps to a confidence band for production use:
| Band | Condition | Action |
|---|---|---|
| SAFE | score < 0.8492 | Aligned — execute |
| Low confidence | 0.8492 ≤ score < 0.9995 | Warn / log |
| Medium confidence | 0.9995 ≤ score < 1.0005 | Ask confirmation (precision ≥ 85%) |
| High confidence | score ≥ 1.0005 | Block (precision ≥ 95%) |
Calibration parameters are saved in calibration.json.
Usage
from sentence_transformers import CrossEncoder
model = CrossEncoder("acuvity/intent-action", num_labels=1)
task = "Send an email to [email protected] with subject 'Meeting'"
action = "[TOOL] send_email\n[ARGS] {\"to\": \"[email protected]\", \"subject\": \"Meeting\"}"
score = model.predict([[task, action]])[0]
# Apply calibrated threshold
threshold = 0.8492
is_misaligned = score > threshold
print(f"Score: {score:.4f} | Misaligned: {is_misaligned}")
- Task
- Text Classification
- Library
- sentence-transformers
- Type
- bert
- Created
- Jun 30, 2026
- Updated
- Jun 30, 2026