data-to-text-executive-report-generator-flan-t5-base

Data-to-Text Executive Report Generator — FLAN-T5 Base

A controlled table-to-text generation model for converting CSV/JSON KPI tables into concise executive reports.

The system combines a fine-tuned FLAN-T5 Base generator with deterministic KPI analysis, row-level source references, structural normalization, and a numerical claim verifier. Unsupported model claims are blocked or deterministically replaced before they are displayed.

Published artifact: merged FLAN-T5 Base fine-tuned model used by Project 04.

Live Resources

• GitHub Project: https://github.com/unit-mole/encoder-decoder-projects/tree/main/04-data-to-text-executive-report-generator
• Hugging Face Model: https://huggingface.co/anmol-unitmole/data-to-text-executive-report-generator-flan-t5-base
• Live Demo: Vercel deployment will be added after deployment is completed.

---

Evaluation Overview

The final PyTorch model was evaluated on 250 held-out KPI-to-report examples.

The application does not trust raw generated claims blindly. Model-generated claim plans pass through a deterministic verifier that checks row references and numerical support before final display.

Final 250-Example Test Results

Factuality and Verification Results

What These Results Mean

The raw generator is not treated as a standalone factual authority. It is intentionally deployed behind a numerical/source verifier.

The most important deployment result is:

100% final displayed claim support on the 250-example evaluation after verifier gating.

The verifier prevented unsupported raw claims from reaching the final report while preserving complete section coverage.

---

ONNX Optimization

The final model was exported to ONNX and an INT8 quantized version was produced.

Runtime Smoke Evaluation

The PyTorch result is the full 250-example quality evaluation. The ONNX rows are deliberately small 5-example runtime/parity smoke tests, not statistically equivalent quality benchmarks.

Quantization

In this local evaluation environment, ONNX CPU-style generation was slower than CUDA PyTorch generation. ONNX is therefore retained as an export/optimization artifact rather than being presented as a latency improvement.

---

Training Summary

The final published checkpoint is the merged model and can be loaded directly with transformers.

---

Dataset and Verification

The project generated and validated a synthetic business-KPI corpus designed for controlled table-to-text generation.

The final fine-tuning run used a 6,000-example subset of the verified training split.

---

System Architecture

CSV / JSON KPI Table
        ↓
Parsing + Validation
        ↓
Schema Detection
        ↓
Deterministic KPI Analysis
        ↓
Controlled Prompt + Fact Index
        ↓
Fine-Tuned FLAN-T5 Base
        ↓
Lean Structured Claim Plan
        ↓
Structural Normalization
        ↓
Deterministic Fact Hydration
        ↓
Numerical / Source Claim Verifier
        ↓
Unsupported Claim Blocking / Replacement
        ↓
Verified Executive Report

The model generates a compact claim plan containing section codes, claim types, source row IDs, and report text. Numerical values and factual fields can then be reconstructed from verified source rows before final claim validation.

---

Why a Verifier-Gated Architecture?

A fluent generated sentence can still contain an unsupported number or attribution.

For executive KPI reporting, the project therefore separates:

1. Language generation — FLAN-T5 determines how to express the report.
2. Deterministic analysis — KPI movements and source facts are calculated outside the model.
3. Evidence mapping — generated claims reference source rows.
4. Verification — unsupported claims are blocked.
5. Hydration/replacement — supported factual content is reconstructed when necessary.

This design prioritizes final report factuality over trusting unconstrained generation.

---

Example Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

repo_id = "anmol-unitmole/data-to-text-executive-report-generator-flan-t5-base"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)

prompt = """
Generate a structured executive KPI claim plan from the supplied
table analysis and source-row references.
"""

inputs = tokenizer(
    prompt,
    return_tensors="pt",
    truncation=True,
    max_length=1280,
)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    num_beams=1,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The full Project 04 application performs additional deterministic hydration and claim verification after model generation. For production-style use, use the complete GitHub pipeline rather than treating the raw model output as the final verified report.

---

Model Selection and Deployment Decision

The final candidate was selected because it demonstrated:

• strong text similarity and semantic quality,
• near-perfect structured claim-plan extraction,
• complete final section coverage,
• deterministic source-row traceability,
• verifier interception of unsupported claims,
• 100% final displayed claim support in the 250-example evaluation,
• successful FP32 ONNX export,
• successful INT8 ONNX quantization and runtime smoke validation.

The project does not claim that raw FLAN-T5 generations are perfectly factual. The deployment design explicitly relies on the verifier.

---

Experimental Environment

Training and evaluation were executed locally using:

---

Quality Gates

---

Intended Use

This model and project are suitable for:

• KPI table summarization experiments
• executive-report generation demonstrations
• controlled table-to-text generation research
• source-grounded generation workflows
• numerical-verification experiments
• ONNX export and quantization experiments
• machine-learning portfolio demonstrations
• educational use

---

Limitations

• Raw model generations can contain unsupported claims before verifier gating.
• Raw numerical factuality is substantially lower than final verifier-gated factuality.
• The current dataset is synthetic/artificial business KPI data and does not represent every real enterprise reporting format.
• ONNX quality was smoke-tested on 5 examples; the 250-example full evaluation was performed with the PyTorch model.
• ONNX generation was slower than CUDA PyTorch generation in the measured local environment.
• The model should not be used as a standalone authority for financial, legal, compliance, medical, safety-critical, or other high-stakes decisions.
• Human review remains appropriate for consequential reporting.

---

Responsible Use

The model is intended for educational, portfolio, research, analytics-assistance, and general business-reporting experiments.

Generated reports should be reviewed before consequential use. The complete application includes verification specifically because fluent language generation alone is not sufficient evidence of factual correctness.

---

Reproducibility

The GitHub project contains code for:

• synthetic KPI dataset generation,
• leakage and overlap validation,
• challenge-set generation,
• FLAN-T5 LoRA fine-tuning,
• structured claim-plan generation,
• structural normalization,
• numerical/source verification,
• unsupported-claim blocking,
• content-selection evaluation,
• numerical factuality evaluation,
• hallucination-rate evaluation,
• BLEU / ROUGE-L / BERTScore evaluation,
• ONNX export,
• INT8 quantization,
• runtime validation,
• automated Python tests,
• Next.js application code,
• GitHub Actions validation.

Complete implementation:

https://github.com/unit-mole/encoder-decoder-projects/tree/main/04-data-to-text-executive-report-generator

---

License

Apache License 2.0.