Legal-Qwen3.5-9B-Abliterated

Legal-Qwen3.5-9B-Abliterated

A legal-domain fine-tune of huihui-ai/Huihui-Qwen3.5-9B-abliterated (an abliterated Qwen3.5-9B vision-language model), trained on public-domain United States court opinions from CourtListener. Smaller, faster sibling of Legal-Qwen3.6-27B-Abliterated.

All base capabilities are preserved and were verified after merging: vision, tool/function calling, and thinking mode.

Training

• Method: LoRA (r=32, q/k/v/o/gate/up/down) via Unsloth on 1x RTX 5090; merged into the pristine bf16 base afterward
• Data: CourtListener opinions (CPT) + holding-summary pairs (SFT) + 41 recent 2025-2026 rulings including Chatrie v. United States (S. Ct., June 29, 2026), 2,048-token sequences, 1 epoch

IMPORTANT: pair it with retrieval

Fine-tuning teaches doctrine and style, not verbatim recall. The model can hallucinate reporter citations, dates, and quotes. For real legal-research use, run it with RAG over the CourtListener bulk data (or the linked training corpus) and verify every citation at the source.

Variants

Speed notes from testing on an RTX 5090 (single-stream): ~47 tok/s bf16 in vLLM with CUDA graphs off, 87 tok/s with graphs on, 146 tok/s with an FP8 W8A8 quant (llm-compressor FP8_DYNAMIC, recipe in this card's discussion) plus graphs; ~104 tok/s as Q8_0 in Ollama. Installing the Triton-based flash-linear-attention package speeds up the hybrid attention layers in vLLM.

Warnings

• Abliterated base: refusal behaviors of the original Qwen3.5 have been removed upstream. You are responsible for output filtering appropriate to your deployment.
• Not legal advice. Outputs are drafts/research aids and must be reviewed by a licensed attorney.
• US-centric: training data is exclusively United States case law.