persian-legal-rag

Persian Legal RAG — three blueprints, zero dependencies

Retrieval-augmented legal QA over Iranian + EU law, built so that no statute can be presented as law unless it is law.

Companion dataset: sosa123454321/persian-legal-kb

Why RAG and not fine-tuning

Iranian law moved in the two weeks before this was built. قانون ملی توسعه هوش مصنوعی was enacted 1405-05-28. لایحهٔ حفاظت از داده‌های شخصی was approved by cabinet in 1403-04 then returned to the legal commission in 1403-07 — it is not law, though most secondary sources still describe it as if it were. Iran's base immigration statute is still from 1310 (1931).

Knowledge baked into weights goes stale silently. Knowledge in a retrieval index is updated by editing a file, and every answer carries its source. Stanford HAI (2025) measured RAG at 28–41% better factual recall and ~35% lower hallucination versus fine-tuning the same base model across 12 knowledge tasks.

The three blueprints

A returns verbatim statute text with article number and source, or refuses. Nothing is generated, so nothing can be invented.

B fuses BM25 (lexical, exact article numbers) with TF-IDF cosine over character trigrams (semantic, robust to irregular Persian morphology) using Reciprocal Rank Fusion, then passes results through a legal-status gate that injects validity warnings into the prompt, then enforces citation with a hallucination guard.

C builds an 80-node / 93-edge graph — Law, Article, Bill, Amendment, Authority, RiskTier, Concept — with typed edges amends, supersedes, proposes_amendment_to, pending_before, maps_to. This answers the multi-hop questions vector search structurally cannot:

«سرنوشت ماده ۱۵ قانون ورود و اقامت چه شده؟»
art:1310-15 ←proposes_amendment_to— bill:reform-art15-16 [🟥 bill_not_enacted]
     ↑has_article
law:ir-entry-residence-1310 ←amends— amd:1336-band-d-art13 [✅ in_force]

Cross-jurisdiction maps_to edges between Iran's four risk tiers and the EU AI Act's four tiers are all marked mapping_confidence: low — the two schemes are not 1:1 and must not be treated as equivalent.

Run it

python3 a_retrieval_only.py "مهلت ثبت اقامت پس از ورود"
python3 a_retrieval_only.py --selftest

python3 b_hybrid_rag.py --selftest
python3 b_hybrid_rag.py --retrieve-only "شرایط اقامت هفت ساله"   # inspect the LLM context
HF_TOKEN=hf_... LLM_PROVIDER=hf LLM_MODEL=google/gemma-3-27b-it \
  python3 b_hybrid_rag.py "لایحه حفاظت از داده های شخصی قانون است؟"

python3 c_kag_graph.py --selftest
python3 c_kag_graph.py --trace art:1310-15
python3 c_kag_graph.py --hop art:bmo-24 2

Generation adapters: hf (HF Inference Providers router), gemini, openrouter, ollama. All retrieval, fusion, gating and graph code is stdlib-only and runs on 2 CPU cores / 1 GB RAM.

Verified behaviour (real end-to-end runs)

Tested against google/gemma-3-27b-it via the HF router:

Two real bugs were found and fixed by these end-to-end runs; see CRITIQUE.md in the dataset repo.

What is NOT done

• The multi-model benchmark is incomplete. eval_models.py is written and working, but the HF account hit HTTP 402 — monthly included credits depleted after 3 successful calls. There is therefore no valid model ranking. The script now refuses to print one below n=10 (exit code 2) precisely so nobody quotes an accuracy computed over 3 samples.
• 6 of 10 planned legal domains are seeded; books and videos are not collected at all.
• Retrieval self-test scores are self-reported on queries written by the same author who wrote the corpus. They measure lexical pipeline correctness, not generalisation.
• No fine-tuned model, no LoRA adapter — no GPU was available in the build environment.

Disclaimer

این خروجی اطلاعات حقوقی است، نه مشاورهٔ حقوقی. Legal information, not legal advice.