qwen25-tax-3b-v3-adapters

qwen25-tax-3b-v3 (LoRA Adapters)

LoRA adapter weights from all three training stages of the v3 IRS Tax Code RL model.

Model Description

Base model: Qwen/Qwen2.5-3B-Instruct Fine-tuning pipeline: SFT → DPO → GRPO Training data: IRC Title 26 (Internal Revenue Code), U.S. Code of Federal Regulations Title 26

This is v3 of the IRS Tax Code RL project.

Training Stages

1. SFT (Supervised Fine-Tuning)

• Adapters in: sft/
• Grounded on IRC Title 26 question-answer pairs
• Multiple checkpoints retained (0000200_adapters.safetensors through 0001000_adapters.safetensors)
• Final adapter: sft/adapters.safetensors

2. DPO (Direct Preference Optimization)

• Adapters in: dpo/
• Trained on preference data generated from SFT model outputs
• Best checkpoint: dpo/adapters_best.safetensors

3. GRPO (Group Relative Policy Optimization)

• Adapters in: grpo/
• RL fine-tuning with citation accuracy and answer correctness reward signals
• Best checkpoint: grpo/adapters_best.safetensors

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")

# Load the final GRPO adapter (recommended)
model = PeftModel.from_pretrained(base, "dennisonb/qwen25-tax-3b-v3-adapters/grpo")

Intended Use

This model is intended for research and educational purposes related to U.S. tax law (IRC Title 26). It is NOT a substitute for professional tax advice.