ClauseQwen

ClauseQwen - Qwen3 Legal Contract Clause Analysis

QLoRA fine-tuning of Qwen3-1.7B for Legal commercial contract-clause analysis using CUAD and LegalBench / ContractNLI.

Legal Commercial contracts contain dense legal language that can make clause review and structured extraction slow when performed manually. ClauseQwen is designed to reduce that effort by identifying clause types, detecting whether contractual concepts are present, extracting relevant language, and returning natural language.

Why QLoRa

QLoRA adds 4-bit base model quantization to reduce memory usage, making it more optimal (on the memory side) than vanilla LoRa.

Model

• Base model: Qwen/Qwen3-1.7B
• Fine-tuning: QLoRA / LoRA
• Quantization: 4-bit NF4
• Training: TRL SFTTrainer
• Adapters: PEFT LoRA

Datasets

CUAD

theatticusproject/cuad

Used for:

• clause-category classification
• clause-presence detection
• relevant-language extraction

LegalBench / ContractNLI

nguha/legalbench

Adds contract reasoning tasks covering concepts such as confidentiality, limited use, survival of obligations, third-party sharing, compelled disclosure, return of confidential information, and related contractual obligations.

Training Run

training was completed locally on an NVIDIA GeForce RTX 3050 with 6 GB VRAM using 4-bit QLoRA

Adapter output:

outputs/legal-qwen3-1.7b-qlora

Base vs Fine-Tuned Evaluation

held-out evaluation of 100 examples compared the original Qwen3-1.7B against Qwen3-1.7B with the legal QLoRA adapter.

largest improvement was in clause-type classification, increasing from 15% to 92%. Extraction F1 increased from 0.7275 to 0.9059 while maintaining 100% valid JSON output.

ContractNLI note: The reported 100-example benchmark contained CUAD extraction and classification examples only. ContractNLI classification accuracy was therefore not evaluated in this benchmark and should be measured separately with a stratified ContractNLI evaluation.

HF Imports

pipeline:

from transformers import pipeline

pipe = pipeline("text-generation", model="AaronTekle/legal-qwen3-1.7b-qlora")

load model directly:

from transformers import AutoModel
model = AutoModel.from_pretrained("AaronTekle/legal-qwen3-1.7b-qlora", device_map="auto")