Downloads · 30 days
17
22% of all-time downloads
AmirMohseni/modernbert-large-v3-seeks-guidance-user-len4096-seed42
modernbert-large-v3-seeks-guidance-user-len4096-seed42 is a text classification model from AmirMohseni. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
Paper: Legal Guidance in the Wild | Code: GitHub
Downloads · 30 days
17
22% of all-time downloads
All-time downloads
76
Public
Parameters
396M
1.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.6 GB · 100%
From the Hugging Face model README
Paper: Legal Guidance in the Wild | Code: GitHub
This checkpoint predicts whether a conversation's user is seeking legal guidance. It is the first stage of the full-conversation ModernBERT-large cascade from the Legal Guidance in the Wild study. Input contains chronological user messages only; assistant messages are excluded. The paired topic model is modernbert-large-v3-primary-topic-user-len4096-seed42.
This is a research routing model, not a legal-advice system. It must not be used to decide whether a person has a valid claim or needs professional counsel.
False: no legal guidance soughtTrue: legal guidance soughtmain (latest at run time)6ba4c2696e889276403fe118d76c8360Dataset access follows the linked repository's sharing settings. The
fingerprints identify the exact loaded splits even if main later changes.
| Setting | Value |
|---|---|
| Base model | answerdotai/ModernBERT-large |
| Maximum length | 4096 |
| Epochs | 3 |
| Learning rate | 5e-05 |
| Effective batch size | 32 |
| Weight decay | 0.01 |
| Seed | 42 |
| Hardware | NVIDIA A100-SXM4-40GB |
The checkpoint was selected by validation macro-F1, followed by guidance
threshold selection. The latest complete run selected threshold 0.38.
| Evaluation | Macro-F1 | Positive F1 | Accuracy |
|---|---|---|---|
| Guidance stage (N=290) | 0.893 | 0.887 | -- |
| Full large cascade (15-way, N=290) | 0.536 | -- | 0.776 |
The paired topic checkpoint obtains 0.716 accuracy and 0.574 macro-F1 on the 134 silver guidance-positive validation conversations. Gold-set results are reported below.
Scored once on the 200 human-adjudicated gold conversations after all modeling decisions were frozen (paper, Tables 4 and 6; per-example predictions are in the GitHub repository).
| Evaluation | Accuracy | Macro-F1 |
|---|---|---|
| Guidance (N=200) | 0.845 | 0.845 |
| Topic, gold guidance-positive (N=84) | 0.762 | 0.671 |
| Full large cascade (N=200) | 0.755 | 0.593 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "AmirMohseni/modernbert-large-v3-seeks-guidance-user-len4096-seed42"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
# conversation: list of {"role", "content"} messages; only user turns are used.
user_only_conversation = "\n".join(
f"User: {m['content']}" for m in conversation if m["role"] == "user"
)
inputs = tokenizer(user_only_conversation, return_tensors="pt", truncation=True,
max_length=4096)
logits = model(**inputs).logits
probability = logits.softmax(-1)[0, model.config.label2id["True"]].item()
seeks_guidance = probability >= 0.38
The threshold 0.38 is the frozen validation-selected operating point and is
the value used for every reported result. The repository's class labels alone
do not encode it, so an argmax over the two logits will not reproduce the
cascade predictions.
The checkpoint was trained on English-language public LLM interaction logs with silver labels, one source, and one seed. Inputs beyond 4,096 tokens are truncated. The dataset is jurisdiction-agnostic, may contain sensitive content, and is not representative of all legal-help seekers. Calibration and accuracy may shift across platforms, populations, jurisdictions, and time. Human review is required for consequential use.
@inproceedings{mohseni2026legal,
title = {Legal Guidance in the Wild: How Users Seek Legal Help in Real-World {LLM} Conversations},
author = {Mohseni, Amirreza and Spanakis, Gerasimos},
booktitle = {Legal Knowledge and Information Systems (JURIX 2026)},
year = {2026},
eprint = {2610.25159},
archivePrefix = {arXiv}
}