Downloads · 30 days
21
26% of all-time downloads
AmirMohseni/modernbert-base-v3-seeks-guidance-user-len4096-seed42
modernbert-base-v3-seeks-guidance-user-len4096-seed42 is a text classification model from AmirMohseni. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
Paper: Legal Guidance in the Wild | Code: GitHub
Downloads · 30 days
21
26% of all-time downloads
All-time downloads
81
Public
Parameters
150M
1.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors598 MB · 99%
From the Hugging Face model README
Paper: Legal Guidance in the Wild | Code: GitHub
This checkpoint predicts whether a conversation's user is seeking legal guidance. It is the first stage of the full-conversation ModernBERT-base cascade from the Legal Guidance in the Wild study. Input contains chronological user messages only; assistant messages are excluded. The second-stage topic model is modernbert-base-v3-primary-topic-user-len4096-seed42.
This is a research routing model, not a legal-advice system. It must not be used to decide whether a person has a valid claim or needs professional counsel.
False: no legal guidance soughtTrue: legal guidance soughtmain (latest at run time)6ba4c2696e889276403fe118d76c8360Dataset access follows the linked repository's sharing settings. The
fingerprints identify the exact loaded splits even if main later changes.
| Setting | Value |
|---|---|
| Base model | answerdotai/ModernBERT-base |
| Maximum length | 4096 |
| Epochs | 3 |
| Learning rate | 5e-05 |
| Effective batch size | 32 |
| Weight decay | 0.01 |
| Seed | 42 |
| Hardware | NVIDIA A100-SXM4-40GB |
The checkpoint was selected by validation macro-F1, followed by guidance
threshold selection. The latest complete run selected threshold 0.26.
| Evaluation | Macro-F1 | Positive F1 | Accuracy |
|---|---|---|---|
| Guidance stage (N=290) | 0.879 | 0.875 | -- |
| Full base cascade (15-way, N=290) | 0.550 | -- | 0.762 |
The paired topic checkpoint obtains 0.724 accuracy and 0.603 macro-F1 on the 134 silver guidance-positive validation conversations. Gold-set results are reported below.
Scored once on the 200 human-adjudicated gold conversations after all modeling decisions were frozen (paper, Tables 4 and 6; per-example predictions are in the GitHub repository).
| Evaluation | Accuracy | Macro-F1 |
|---|---|---|
| Guidance (N=200) | 0.815 | 0.813 |
| Topic, gold guidance-positive (N=84) | 0.750 | 0.646 |
| Full base cascade (N=200) | 0.725 | 0.544 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "AmirMohseni/modernbert-base-v3-seeks-guidance-user-len4096-seed42"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
# conversation: list of {"role", "content"} messages; only user turns are used.
user_only_conversation = "\n".join(
f"User: {m['content']}" for m in conversation if m["role"] == "user"
)
inputs = tokenizer(user_only_conversation, return_tensors="pt", truncation=True,
max_length=4096)
logits = model(**inputs).logits
probability = logits.softmax(-1)[0, model.config.label2id["True"]].item()
seeks_guidance = probability >= 0.26
The checkpoint was trained on English-language public LLM interaction logs with silver labels, one source, and one seed. Inputs beyond 4,096 tokens are truncated. The dataset is jurisdiction-agnostic, may contain sensitive content, and is not representative of all legal-help seekers. Calibration and accuracy may shift across platforms, populations, jurisdictions, and time. Human review is required for consequential use.
@inproceedings{mohseni2026legal,
title = {Legal Guidance in the Wild: How Users Seek Legal Help in Real-World {LLM} Conversations},
author = {Mohseni, Amirreza and Spanakis, Gerasimos},
booktitle = {Legal Knowledge and Information Systems (JURIX 2026)},
year = {2026},
eprint = {2610.25159},
archivePrefix = {arXiv}
}