octen-law-8b-v2

Octen Law 8B v2

Octen Law 8B v2 is a legal-domain dense embedding model from Litil Labs. It is based on nvidia/Nemotron-3-Embed-8B-BF16 and post-trained on public legal citation data for first-stage retrieval.

Evaluated model revision: de09ddfc3911e92a2e053b1a1a2e34e60d624287

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "litillabs/octen-law-8b-v2",
    revision="de09ddfc3911e92a2e053b1a1a2e34e60d624287",
)

queries = model.encode(
    ["query: What authority governs ...?"],
    normalize_embeddings=True,
)
documents = model.encode(
    ["passage: The court held that ..."],
    normalize_embeddings=True,
)
scores = queries @ documents.T

Use the query: prefix for queries and the passage: prefix for documents. The model uses attention-mask mean pooling and L2-normalized embeddings. The published maximum sequence length is 32,768 tokens.

Training

The released checkpoint was trained for 200 optimizer updates with LoRA and then merged into the base model and exported as BF16 safetensors.

• Objective: set-valued InfoNCE with temperature 0.02
• Negatives: four query-specific explicit negatives per example
• LoRA: rank 8, alpha 16, dropout 0 on q_proj, k_proj, v_proj, and o_proj
• Optimizer: AdamW with peak learning rate 1e-5, weight decay 0.01, cosine decay, and 10% warmup
• Logical batch size: 4
• Training length: 200 optimizer updates
• Seed: 17

Training data

Post-training used 10,000 examples built from the public theresiavr/legalpincite dataset at revision ebf0c37cca4c8b69291fb9eecc8ecb8784575637.

LegalPincite is released under CC BY 4.0 with an EUR-Lex reuse notice. The model weights are distributed under the NVIDIA OpenMDW License Agreement 1.1 and the notices included in this repository.

Evaluation

The checkpoint was selected on three legal retrieval datasets. The base model and candidate used the same evaluation runtime.

Limitations

• Training data are English EU legal-citation examples, so performance may vary across languages and jurisdictions.
• Dense retrieval can miss exact citations, rare names, dates, and lexical constraints.
• The model has not been evaluated as a legal-advice or outcome-prediction system.

Citation

@misc{litillabs_octen_law_8b_v2_2026,
  title  = {Octen Law 8B v2},
  author = {Litil Labs},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/litillabs/octen-law-8b-v2}}
}