scribegene-llm-v0.4

MDF Form Reader — Qwen2-VL-7B Fine-tuned

Vision-native handwritten insurance form understanding, fine-tuned from unsloth/Qwen2-VL-7B-Instruct using QLoRA.

No OCR needed. This model reads handwriting, checks checkbox states, and extracts structured data directly from scanned MDF (Monthly Disability Verification) form images.

---

 Model Summary

---

 Quick Start

from transformers import AutoModelForCausalLM, AutoProcessor
from PIL import Image
import torch

model_id = "solvrays/scribegene-llm-v0.4"

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    trust_remote_code=True,
)

# Load your scanned MDF form image
image = Image.open("mdf_form.png").convert("RGB")

# Ask a question about the form
question = "What is the name of the physician who signed this form?"

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": question}
    ]
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = processor(text=[text], images=[image], return_tensors="pt").to("cuda")

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=200, temperature=0.1)

answer = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(answer)

---

 What is an MDF Form?

A Monthly Disability Verification Form (Form 441.O.MDF.O) is issued by TriPlus Services, acting as Third-Party Administrator of Penn Treaty Network America and American Network policies. It requires a licensed physician to certify a patient's ongoing disability status monthly.

Key Fields Extracted

• Physician name, address, phone, fax
• Submission date range (from / to)
• Patient disability status (YES checked / NO checked)
• Disability end date (if applicable)
• Form completion date
• Physician signature presence

---

 Why Vision-Native vs OCR?

---

 Training Pipeline

Scanned MDF Form (PDF)
    ↓ Image pre-processing (deskew 300 DPI, bilateral denoise, CLAHE)
    ↓ Vertex AI Gemini 2.5 Flash → structured JSON annotation
    ↓ VQA triplet dataset (field extraction + OOD refusal pairs)
    ↓ Qwen2-VL-7B + QLoRA (Unsloth, 2-5× faster, 80% less VRAM)
    ↓ Merge adapters → full 16-bit model
    ↓ HuggingFace Hub (safetensors)

Training Configuration

base_model: unsloth/Qwen2-VL-7B-Instruct
fine_tuning_method: QLoRA (NF4, double quantization)
lora_rank: 16
lora_alpha: 32
lora_dropout: 0.05
use_rslora: true
vision_layers: frozen
language_layers: adapted
optimizer: AdamW 8-bit (paged)
lr_scheduler: cosine
neftune_noise_alpha: 5
annotator: Vertex AI Gemini 2.5 Flash
framework: Unsloth + HuggingFace TRL

---

 Evaluation Results

OOD Refusal Rate measures how reliably the model declines to answer questions not answerable from the form (e.g. "What is the diagnosis?", "Has this claim been approved?").

---

⚠️ Limitations

• Domain-specific: Trained exclusively on TriPlus Services MDF forms. Performance on other form types is not guaranteed.
• Image quality: Works best on scans ≥ 300 DPI. Very low-resolution or heavily degraded scans may reduce accuracy.
• Language: English only.
• Redacted fields: Returns null for blacked-out fields (insured name/policy number).
• Not for medical diagnosis: This model extracts administrative form data only.

---

 License

This model is released under the Apache 2.0 License. The base model (unsloth/Qwen2-VL-7B-Instruct) is also Apache 2.0.

---

 Acknowledgements

• Unsloth for 2-5× faster fine-tuning
• Qwen2-VL for the base vision-language model
• Vertex AI Gemini 2.5 Flash for dataset annotation
• HuggingFace TRL for SFTTrainer