vi-legal-query-rewriter-lora

vi-legal-query-rewriter-lora

A LoRA adapter that rewrites colloquial Vietnamese legal questions into formal statutory queries, built as the input node of a RAG system over 242,063 Vietnamese statutory articles. Published together with the retrieval-level evaluation used to assess it.

Scope note: on the held-out test set this adapter did not improve retrieval — passing the user's question through unchanged scored higher. It is released for reproducibility of the evaluation and for the diagnostic findings below, not as a production component.

Tiếng Việt: Adapter LoRA viết lại câu hỏi pháp luật thông thường thành truy vấn pháp lý chuẩn, dùng làm node đầu vào cho hệ RAG trên 242.063 điều luật Việt Nam. Trên tập test độc lập, adapter không cải thiện truy hồi so với dùng nguyên câu hỏi gốc, nên nó được công khai để tái lập phép đo và để chia sẻ các phát hiện kỹ thuật bên dưới.

---

The problem it addresses

Vietnamese legal search has a wide vocabulary gap between how people ask and how statutes are written:

Closing that gap by rewriting the query is a natural design; this repo records what happened when it was built and measured properly.

Evaluation

Held-out test: 122 questions, split by source document, so no document appears in both train and test. Retrieval = hybrid dense + BM25 with RRF fusion.

Prompting the base model is roughly neutral against the no-op baseline; the fine-tuned adapter scores below both.

These absolute numbers come from a retrieval configuration with reranking and the legal-hierarchy prior disabled, so they understate the host system — the same question set scores R@1 0.376 through the full pipeline. The three rows above are measured under identical conditions, so the comparison between them holds.

Diagnosis: the training target, not the training run

Training pairs were generated by distillation — a 35B model read each statutory article and wrote both a colloquial question and a "proper" legal query for it. Because the teacher could see the article, it wrote queries the way a librarian would: by naming the document.

Measured on the training set itself:

What the adapter therefore learned to produce:

input : Khi cửa khẩu bị tạm đóng vì an ninh hay dịch bệnh, những ai được quyền
        ra quyết định và tối đa bao lâu?
output: Luật Sửa đổi, bổ sung một số điều của Luật Biên phòng về cửa khẩu

The retriever matches on article body text, not document titles. Targets that discard 75% of the content words — the exact signal dense retrieval uses — and substitute a document-type name remove more information than they add.

A second training round addressed this directly: targets were rebuilt algorithmically from the question's own content words plus tf-idf-selected legal terminology from the article, raising content-word retention from 25% to 100% and eliminating document references entirely. Retrieval quality fell further (R@1 0.072). That result points at the premise rather than the data: the original question is already fluent Vietnamese, which is the distribution the bi-encoder was trained on, so a rewrite that reduces fluency has little room to help.

Findings worth reusing

Epoch-boundary memorisation is visible in the loss curve. Training loss dropped abruptly from 1.607 to 0.831 at step 240. With 894 pairs at batch size 4, one epoch is ~224 steps — so step 224 is where epoch 2 begins and the model starts seeing the same data again. The train/validation gap then widened to nearly 3× (0.544 vs 1.586) while validation loss flattened. One epoch would have been sufficient for this set.

mlx-lm defaults to mask_prompt: false, computing loss over the prompt as well as the completion. For a rewriting task the model should learn to write the query, not to ask the question — this config sets mask_prompt: true.

Validation loss is not a proxy for retrieval quality. Loss fell from 4.567 to 1.586 across the run. Recall@k is the metric that decides whether a rewriting node earns its place, and it moved in the opposite direction.

Training configuration

Usage

from mlx_lm import load, generate
model, tok = load("mlx-community/Qwen2.5-7B-Instruct-4bit",
                  adapter_path="path/to/this/adapter")

If you are building this node

1. Build the retrieval-level evaluation before training. It is the only signal that answers whether the node helps.
2. Do not let the teacher model see the article when generating rewriting targets, or it will produce document references instead of content paraphrases.
3. Keep the question's content words. In the raw distillation output, 66% of targets contained an actual document number; training on those teaches citation fabrication — the most damaging failure mode for a legal tool.
4. Check whether the node is needed at all. With cross-encoder reranking and a legal-hierarchy prior, the host system reaches R@1 0.870 on a hand-verified question set with no rewriting. When retrieval is already strong, an intermediate transformation has more information to lose than to add.

Data & attribution

Training pairs were distilled from Vietnamese statutory text sourced from vbpl.vn — the legal document portal of the Ministry of Justice of Viet Nam — via the CC BY 4.0 dataset th1nhng0/vietnamese-legal-documents. Original statutory text belongs to the Ministry of Justice of Viet Nam.

Adapter weights are released under Apache-2.0. Base model terms (Qwen2.5-7B-Instruct) apply to any derived use.

Not legal advice

Neither this adapter nor the system it was built for constitutes legal advice. Statutory lookup output must always be checked against the full text of the cited provision.