modernbert-large-v3-primary-topic-user-len4096-seed42

ModernBERT-large for primary legal-topic classification

Paper: Legal Guidance in the Wild | Code: GitHub

This checkpoint assigns a primary legal topic to conversations in which the user is seeking legal guidance. It is the second stage of the full-conversation ModernBERT-large cascade from the Legal Guidance in the Wild study. Input contains chronological user messages only; assistant messages are excluded. The first-stage detector is modernbert-large-v3-seeks-guidance-user-len4096-seed42.

This is a research classifier, not a legal-advice system. Its topic prediction must not be treated as a determination of jurisdiction, rights, or legal merit.

Labels

The 14 labels are: FAMILY_AND_ESTATES, HOUSING_AND_PROPERTY, EMPLOYMENT_AND_LABOR, IMMIGRATION_AND_CITIZENSHIP, CRIMINAL_LAW, TAX_LAW, CONSUMER_AND_PERSONAL_FINANCE, BUSINESS_AND_COMMERCIAL, TORTS_AND_CIVIL_LIABILITY, GOVERNMENT_AND_ADMINISTRATIVE, CIVIL_RIGHTS_AND_CONSTITUTIONAL, DATA_PRIVACY_AND_TECHNOLOGY, INTELLECTUAL_PROPERTY, and OTHER. The exact ID mapping is stored in config.json.

Data

• Dataset: AmirMohseni/WildChat-Legal-Classification-V3-Hierarchical
• Requested revision: main (latest at run time)
• Train fingerprint: 6ba4c2696e889276
• Validation fingerprint: 403fe118d76c8360
• Topic-stage train / validation rows: 750 / 134 guidance-positive conversations
• Input mode: chronological user messages only

Dataset access follows the linked repository's sharing settings. The fingerprints identify the exact loaded splits even if main later changes.

Training configuration

The checkpoint was selected by silver-validation macro-F1.

Silver-validation results

The paired first-stage guidance detector obtains 0.893 macro-F1 and 0.887 positive-class F1. Gold-set results are reported below.

Gold-set results

Scored once on the 200 human-adjudicated gold conversations after all modeling decisions were frozen (paper, Tables 4 and 6; per-example predictions are in the GitHub repository).

Inference

from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo = "AmirMohseni/modernbert-large-v3-primary-topic-user-len4096-seed42"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)

# Run only after the paired guidance detector predicts guidance-seeking.
user_only_conversation = "\n".join(
    f"User: {m['content']}" for m in conversation if m["role"] == "user"
)
inputs = tokenizer(user_only_conversation, return_tensors="pt", truncation=True,
                   max_length=4096)
topic_id = int(model(**inputs).logits.argmax(-1)[0])
topic = model.config.id2label[topic_id]

Limitations

The model was trained on English-language public LLM interaction logs with silver labels, one source, and one seed. Several topics have limited validation support, so macro-F1 is unstable and topic-specific errors can be substantial. Inputs beyond 4,096 tokens are truncated. The taxonomy is jurisdiction-agnostic and cannot substitute for legal triage by a qualified professional. Human review is required for consequential use.

Citation

@inproceedings{mohseni2026legal,
  title         = {Legal Guidance in the Wild: How Users Seek Legal Help in Real-World {LLM} Conversations},
  author        = {Mohseni, Amirreza and Spanakis, Gerasimos},
  booktitle     = {Legal Knowledge and Information Systems (JURIX 2026)},
  year          = {2026},
  eprint        = {2610.25159},
  archivePrefix = {arXiv}
}