mistral-insurance-ticket-formatter

Mistral 7B - Insurance Ticket Formatter (LoRA adapter)

A LoRA adapter for Mistral-7B-Instruct-v0.2 that converts informal, messy stakeholder requests into structured engineering tickets (Title + Description format), specialized for insurance industry workflows — claims, underwriting, billing, compliance, and policy admin.

Training details

• Base model: mistralai/Mistral-7B-Instruct-v0.2
• Method: LoRA (r=16, alpha=32, target modules: q/k/v/o_proj) via PEFT, 4-bit NF4 quantized (QLoRA-style) for training on a free Colab T4 GPU
• Trainable parameters: 13.6M / 7.26B total (0.19%)
• Training data: 15 hand-crafted instruction/response pairs covering real insurance domain scenarios
• 6 optimizer steps, 3 epochs — training loss dropped from 3.60 to 1.92

Before / after example

Input: "the system doesnt stop someone from binding a policy in a state we're not even licensed to sell in"

Base Mistral: Produces a rambling response with inconsistent structure ("Steps to Reproduce," "Expected Result," closing with "We appreciate your prompt response on this critical security concern.")

Fine-tuned:

Title: Unauthorized Policy Binding in Unlicensed State Description: The current system configuration allows users to bind policies in states where our organization is not authorized...

What changed vs. what didn't

Fine-tuning did NOT make the model "smarter" — it changed its behavior: consistent Title/Description structure, terse professional tone, no conversational filler. This is the real distinction between fine-tuning (behavior/style) and RAG (knowledge) — this project isolates and demonstrates that difference directly.

Limitations

• Trained on only 15 examples / 6 optimizer steps as a learning exercise
• Consistently produces Title + Description; the third intended section (Acceptance Criteria) did not reliably appear even with extended generation length — likely needs more training examples or steps to fully lock in the 3-part format
• A production version would use hundreds of real historical tickets