legalcite-explain-7b-lora

legalcite-explain-7b-lora

A QLoRA adapter on Qwen/Qwen2.5-7B-Instruct that writes a short, grounded explanation for a citation-support label. It does not decide the label — the label comes from legalcite-support-base (the primary encoder artifact), upstream of this model in the legalcite pipeline. This adapter's only job is to justify a verdict someone else already reached, from the same evidence that produced it.

Automated verification only. Not legal advice. Every citation flagged or cleared here must be independently confirmed by a licensed attorney before filing. Jurisdiction: US federal and state case law only.

This notice is printed on every output of the legalcite CLI and cannot be suppressed by any flag or config value.

Intended use

Secondary, optional component: given (label, proposition, citation, case_name, cited_text), write one or two sentences explaining why the cited text does or does not support the proposition, consistent with the given label. Intended to make a legalcite finding easier for a human reviewer to audit quickly — not to add independent verification weight.

Do not use this model for: deciding a support label (it doesn't have that capability by design — see "How this model was trained" below), legal advice, drafting argument text, or recommending alternative citations.

Important asymmetry to understand before trusting this model's output: if the upstream label is wrong, this model will still write a fluent, confident-sounding explanation justifying that wrong label — it was trained to explain a given verdict, not to independently check one. A well-written explanation is not evidence that the label is correct.

Usage

Not yet wired into the legalcite CLI. legalcite check currently runs the encoder only — this adapter has to be invoked separately for now, with a label you've already obtained (from the encoder, or otherwise). This is an honest gap, not a design choice: the pipeline's docstring has said "optional decoder explanation" since Stage E was built, but CitationVerifier never actually calls a decoder. Wiring it in is open work.

Loading the adapter

Requires peft, transformers, bitsandbytes, torch. The base model downloads separately from this adapter (~15GB) the first time you run this.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

ADAPTER = "vrushket/legalcite-explain-7b-lora"
BASE_MODEL = "Qwen/Qwen2.5-7B-Instruct"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, ADAPTER)
model.eval()

Generating an explanation

The prompt is ChatML: a fixed system message, then a user turn with the label and evidence fields laid out plainly. Reusing the exact same example verified in legalcite-support-base's model card (the encoder classified it SUPPORTED at 0.99 confidence):

SYSTEM_PROMPT = (
    "You are a citation-verification assistant. A citation-checking system has already "
    "determined the label below. In one or two sentences, explain why the cited text does "
    "or does not support the proposition, given that label. Do not decide or second-guess "
    "the label. Do not give legal advice, and do not suggest an alternative citation."
)

label = "SUPPORTED"
proposition = (
    "The right to privacy, as established in Roe v. Wade, 410 U.S. 113 (1973), "
    "is broad enough to encompass a woman's decision whether to terminate her pregnancy."
)
citation_string = "410 U.S. 113"
case_name = "Roe v. Wade"
cited_text = (
    "The Court held that the right to privacy, whether it be founded in the Fourteenth "
    "Amendment's concept of personal liberty, is broad enough to encompass a woman's "
    "decision whether or not to terminate her pregnancy."
)

user_prompt = (
    f"Label: {label}\nProposition: {proposition}\n"
    f"Citation: {citation_string} ({case_name})\nCited text: {cited_text}"
)
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": user_prompt},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=150, do_sample=False)

explanation = tokenizer.decode(
    output_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True
).strip()
print(explanation)
# The cited text supports this proposition. It states: "The Court held that the right
# to privacy, whether it be founded in the Fourteenth Amendment's concept of personal
# liberty, is broad enough to encompass a..."

Or, with the package installed, reuse the exact tested prompt-building code instead of hand-rolling it:

from legal_citation_check.models.explain import build_chat_messages

messages = build_chat_messages(label, proposition, citation_string, case_name, cited_text)

Note that legal_citation_check.models.explain.load_model_and_tokenizer is the training-time loader (it applies a fresh LoRA config via get_peft_model, it does not load saved adapter weights) — for inference against these published weights, use PeftModel.from_pretrained as shown above, not that function.

How this model was trained

Base model and adapter

Qwen/Qwen2.5-7B-Instruct, 4-bit NF4 quantization (bfloat16 compute dtype), LoRA rank 32 / alpha 64 / dropout 0.05, targeting q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj. Prompt is ChatML-formatted: a system message instructing the model to explain (never decide) the label and never give legal advice, a user turn with the (label, proposition, citation, case_name, cited_text) fields, and an assistant turn with the target explanation. Loss is computed only on the assistant turn (DataCollatorForCompletionOnlyLM).

Why the training targets are templated, not human-written

There is no human-written explanation anywhere in this project's dataset. The fix: examples.jsonl (produced during encoder training data synthesis) already carries a perturbation field recording why a given negative example was constructed (wrong_case_name, fabricated_cite, real_cite_wrong_quote, etc.) — available at training-target-generation time, but never part of what this model actually sees as input, at training or inference. Each label/perturbation combination maps to a literal template ("the cited text does/does not say X"), never a legal conclusion — Stage D's gate explicitly fails any explanation that crosses that line. This means the training signal is grounded and correctly-labeled, but the range of phrasing and reasoning styles it teaches is bounded by what a modest set of templates can express — see "Known limitations" below.

Training data size

15,000 training examples / 2,000 eval examples, uniform-randomly subsampled from the ~356K-example labeled set (a random sample from an already label-balanced set preserves the same label proportions). This is a deliberate resource trade-off, not a data-availability limit: a real training run on the full set measured ~53 seconds per optimizer step regardless of how batch size vs. gradient accumulation was split (same total compute either way), projecting to roughly 70 wall-clock hours against a target of single-digit hours. The subsampled run took 3.56 hours for 3 epochs across 3x RTX 6000 Ada (48GB, PCIe, no NVLink).

Real training bugs fixed along the way

Documented here because they shaped the final training config, not just as trivia: (1) a DDP model-loading race where concurrent from_pretrained calls across ranks intermittently left one process with an empty model, fixed with accelerate's main_process_first() barrier; (2) NCCL's peer-to-peer transport hangs indefinitely on this PCIe-only, no-NVLink hardware (NCCL_P2P_DISABLE=1 required, not optional); (3) DDP + gradient checkpointing's default reentrant autograd is incompatible with a frozen-base/LoRA model (RuntimeError: Expected to mark a variable ready only once), fixed with non-reentrant checkpointing; (4) the originally-specified max_seq_length=4096 was ~2.5x the real data's actual max (measured: p50=1010, p90=1235, p99=1487, max=1603 tokens), causing both a CUDA OOM and much of the excess training time.

Evaluation

Training/eval loss

Eval loss dropped from 0.0063 (epoch 1) to 0.0041 (epoch 3) — consistent with the model learning the templated target patterns well, including on held-out examples (different citations from the training set, same underlying template distribution).

Faithfulness gate (spec: "explanations judged faithful to the

retrieved text on 100 samples; any explanation asserting a legal conclusion beyond 'the cited text does or does not say X' is a failure")

Ran an automated proxy for this gate — not the full human-judgment process the spec describes — on 100 held-out test examples: generated an explanation for each, checked for a fixed list of legal-conclusion-style phrases ("you should", "we recommend", "the law requires", "is unconstitutional", etc.).

Result: 0/100 forbidden-phrase violations. Spot-checked generations were correctly label-consistent and grounded in the actual retrieved text — SUPPORTED explanations quoted real evidence, FABRICATED/ wrong_case_name-style UNSUPPORTED explanations correctly named the mismatch, on examples the adapter had not seen during training.

This is not the same as true human-judged faithfulness. The automated check catches an explicit list of forbidden phrases; it cannot catch a subtler legal-conclusion violation phrased differently, and it says nothing about explanation quality beyond the binary faithfulness question. Genuine human review of the 100-sample gate, as the spec describes, has not yet been done.

Known limitations

1. This model cannot verify its own explanations. It is trained to justify whatever label it's given; a wrong upstream label produces a confidently-wrong explanation, not a flagged disagreement.
2. Templated training targets bound the range of explanation styles. The seven templates (one per label/perturbation combination) are literal and grounded, but not stylistically diverse — real-world explanations for cases not resembling any of the five perturbation types (or genuinely novel SUPPORTED/NOT_VERIFIABLE phrasing patterns) are less certain to generalize well, in the same spirit as the encoder card's documented generalization gap on out-of-distribution proposition phrasing.
3. Trained on 15K of the ~356K available labeled examples, for tractable training time on this hardware — not because more data wouldn't plausibly help; revisiting this with more compute/time budget or a lighter base model is open future work.
4. The faithfulness gate result is an automated proxy, not the human-judged 100-sample review the spec calls for.
5. Requires the base model at inference time. This repository is a LoRA adapter only (~323MB); running it requires Qwen/Qwen2.5-7B-Instruct (~15GB) loaded alongside it, typically in 4-bit.
6. Inherits every limitation of the upstream encoder and existence check — see legalcite-support-base's model card, in particular the documented out-of-distribution generalization gap on synthetically-phrased or question-form propositions, and the CourtListener existence-check rate limit (125 requests/day).

Not legal advice

This tool assists a legal professional in checking their own work. It does not tell anyone what the law is, and does not create an attorney-client relationship. This model writes an explanation of an existing label — never a recommendation, never "you should cite X instead." Every citation flagged or cleared by the legalcite pipeline must be independently confirmed by a licensed attorney before filing.

License

Apache-2.0 (adapter weights). Base model Qwen/Qwen2.5-7B-Instruct is licensed separately by Alibaba Cloud — check its own license before redistribution. Training data: derived from CC0 (Caselaw Access Project) and CourtListener public data; explanation targets are templated, not sourced from any third-party text.