insureos-models

INSUREOS Models — Complete Insurance AI Training Pipeline

Created by Bytical AI — AI agents that run insurance operations.

Start Here

If you are trying to understand the Hugging Face repos, start with LEARN.md.

That guide explains, in plain English, why the project is split across model, dataset, code, and search repos, and it walks through the concepts, data, and training code behind each model in this repository.

Overview

INSUREOS is a complete AI/ML training and inference pipeline for UK insurance operations. This repository contains all source code for data generation, model training, evaluation, data collection, and a hybrid search engine.

Model Suite

Training Dataset

piyushptiwari/insureos-training-data — 10K SFT, 5K DPO, 50K tabular, 10K docs, 8K NER

Repository Structure

insureos-models/
├── data/                        # Synthetic data generation
│   ├── constants.py             # UK insurance constants (regions, perils, regulators)
│   ├── gen_sft.py               # Generate SFT instruction-response pairs
│   ├── gen_dpo.py               # Generate DPO preference pairs
│   ├── gen_documents.py         # Generate insurance documents (12 classes)
│   ├── gen_ner.py               # Generate NER-annotated text
│   ├── gen_tabular.py           # Generate claims tabular data
│   └── generate_all.py          # Run all generators
│
├── collect/                     # Real-world data collection
│   ├── config.py                # Scraping targets and configuration
│   ├── scraper_base.py          # Base HTTP scraper with caching
│   ├── convert_sft.py           # Convert raw docs → SFT/DPO format
│   ├── run_fast.py              # Fast collection orchestrator
│   └── sources/                 # Per-source scrapers
│       ├── wikipedia.py         # Wikipedia insurance articles
│       ├── legislation.py       # UK legislation (legislation.gov.uk)
│       ├── fca.py               # FCA Handbook
│       ├── hf_datasets.py       # HuggingFace insurance datasets
│       ├── rss_news.py          # Insurance news RSS feeds
│       └── education.py         # Insurance education resources
│
├── training/                    # Model training scripts
│   ├── qlora_finetune.py        # QLoRA fine-tuning (Qwen3-4B)
│   ├── dpo_train.py             # DPO alignment training
│   ├── retrain_realworld.py     # Real-world data retraining
│   ├── doc_classifier.py        # ModernBERT document classifier
│   ├── ner_model.py             # ModernBERT NER model
│   ├── fraud_model.py           # XGBoost + Isolation Forest fraud
│   ├── pricing_glm.py           # Tweedie GLM + EBM pricing
│   └── distill.py               # Model distillation (experimental)
│
├── evaluation/                  # Evaluation suite
│   ├── run_eval.py              # Full multi-model evaluation
│   └── results/                 # Evaluation results (JSON)
│
├── search/                      # Hybrid search engine
│   ├── config.py                # Search configuration
│   ├── embedder.py              # BGE-small-en-v1.5 embedding service
│   ├── bm25.py                  # Custom Okapi BM25 implementation
│   ├── vector_store.py          # Qdrant vector store
│   ├── reranker.py              # Cross-encoder reranker
│   ├── hybrid_engine.py         # RRF fusion (vector + BM25 + reranker)
│   ├── indexer.py               # Document ingestion pipeline
│   ├── models.py                # Pydantic data models
│   └── api.py                   # FastAPI REST API
│
├── serve/                       # Model serving
│   └── api.py                   # FastAPI inference endpoints
│
└── scripts/                     # Automation
    ├── setup.sh                 # Environment setup (NVIDIA, Python, deps)
    └── train_all.sh             # Full training pipeline script

Quick Start

1. Environment Setup

# Create virtual environment
python3 -m venv .venv && source .venv/bin/activate

# Install dependencies
pip install torch transformers trl peft bitsandbytes
pip install xgboost scikit-learn interpret
pip install sentence-transformers qdrant-client fastapi uvicorn

2. Generate Training Data

python -m data.generate_all
# Outputs: data/output/ (SFT, DPO, docs, NER, tabular)

3. Train Models

# Train all models sequentially
bash scripts/train_all.sh

# Or individually:
python training/qlora_finetune.py          # InsureLLM QLoRA
python training/dpo_train.py               # InsureLLM DPO
python training/doc_classifier.py          # Document classifier
python training/ner_model.py               # NER model
python training/fraud_model.py             # Fraud detection
python training/pricing_glm.py             # Pricing models

4. Evaluate

python evaluation/run_eval.py
# Results saved to evaluation/results/

5. Run Search Engine

# Index documents
python search/indexer.py

# Start API
python search/api.py
# API at http://localhost:8900
# Endpoints: /search, /search/vector, /search/keyword, /suggest, /facets, /stats

Search Engine — InsureSearch

A hybrid search engine rivaling Azure AI Search, built entirely on open-source components:

Index stats: 33,034 chunks from 31,679 documents, 51,640 BM25 terms.

Training Pipeline

Stage 1: Synthetic Data Generation
├── 10K SFT instruction-response pairs
├── 5K DPO preference pairs
├── 50K tabular claims (Motor/Property/Liability)
├── 10K insurance documents (12 classes)
└── 8K NER-annotated texts (13 entity types)

Stage 2: QLoRA Fine-Tuning → Qwen3-4B
├── rank=64, alpha=128, all-linear targets
├── 2 epochs, batch=2, grad_accum=4
├── Final: train_loss=0.012, eval_loss=0.118
└── Token accuracy: 95.88%

Stage 3: DPO Alignment
├── 5K preference pairs
├── 149 steps, reward_accuracy=1.0
└── Reward margin: 26.76

Stage 4: Real-World Data Collection
├── Wikipedia (150 docs), UK Legislation (692)
├── HuggingFace datasets (31,060), RSS (50), Education (88)
├── Converted to 3,685 SFT + 776 DPO pairs
└── Quality filtered (English-only, no echo responses)

Stage 5: Real-World Retraining
├── 876 steps on real-world SFT data
└── Claims process score improved 0.40 → 0.60

Stage 6: Specialized Models (parallel)
├── FraudNet: XGBoost + Isolation Forest → AUC-ROC 1.0
├── PricingGLM: Tweedie GLM + EBM → MAE £11,132
├── DocClassifier: ModernBERT → F1 1.0
└── InsureNER: ModernBERT → F1 1.0

Tech Stack

• LLM: Qwen3-4B + QLoRA + DPO (PyTorch, Transformers, TRL, PEFT, bitsandbytes)
• Classification & NER: ModernBERT-base (Transformers)
• Fraud Detection: XGBoost + Isolation Forest (scikit-learn)
• Pricing: Tweedie GLM (scikit-learn) + EBM (InterpretML)
• Search: BGE-small-en-v1.5 + Qdrant + BM25 + cross-encoder
• Training GPU: NVIDIA Tesla T4 16GB

Citation

@misc{bytical2026insureos,
  title={INSUREOS: A Complete AI/ML Suite for UK Insurance Operations},
  author={Bytical AI},
  year={2026},
  url={https://huggingface.co/piyushptiwari/insureos-models}
}

About Bytical AI

Bytical builds AI agents that run insurance operations — claims automation, underwriting intelligence, digital sales, and core system modernization for insurers across the UK and Europe. Microsoft AI Partner | NVIDIA | Salesforce.