grammar-correction-flan-t5-base

Grammar Error Correction with FLAN-T5 Base

Fine-tuned FLAN-T5-base encoder-decoder model for English grammatical error correction (GEC), trained and selected through controlled experiments on BEA-2019 W&I+LOCNESS, JFLEG-based evaluation data, controlled synthetic error pairs, and leakage-safe identity examples.

The selected checkpoint is the Base Conservative 30% Identity model. It was chosen because it retained essentially the same ERRANT F0.5 as the strongest Base baseline while improving precision and materially reducing unnecessary edits on clean text.

Selected deployment model: 03c-base-conservative-30pct
Base architecture: google/flan-t5-base
Parameters: 247,577,856
Primary task: English grammar error correction / text-to-text generation

Live Resources

• Model Repository: https://huggingface.co/anmol-unitmole/grammar-correction-flan-t5-base
• Static Demo: https://huggingface.co/spaces/anmol-unitmole/05-grammar-error-correction-t5-encoder-decoder (deployment in progress)
• Base Model: https://huggingface.co/google/flan-t5-base

---

Evaluation Dashboard

The selected model was evaluated using a held-out grammar-correction set plus an independent clean-text over-correction benchmark.

Evaluation Population

The clean metric partition includes the dedicated 988-example clean benchmark plus identity/correct examples already present in the main held-out test data.

---

Model Selection

Multiple controlled fine-tuning experiments were run rather than assuming that a larger model or more aggressive balancing would automatically perform better.

Fine-Tuned Experiment Comparison

Why Base 30% Was Selected

Compared with Base 10%, the selected Base 30% model:

• retained essentially identical ERRANT F0.5: 0.4530 → 0.4527;
• improved precision: 0.4989 → 0.5053;
• improved semantic similarity: 0.9359 → 0.9384;
• reduced clean over-correction: 29.38% → 25.73%;
• improved number preservation: 99.63% → 99.72%;
• slightly improved named-entity preservation: 93.20% → 93.57%.

The cost was a modest recall reduction from 0.3311 to 0.3195.

This made Base 30% the stronger quality-versus-conservatism trade-off for the final portfolio model.

---

Conservative Training Findings

Increasing the proportion of identity/correct examples consistently reduced unnecessary edits in the Small-model ablation:

The Base experiment showed the same direction:

This demonstrated that identity pressure improved conservatism without materially sacrificing the primary F0.5 score.

---

Project Quality Gates

These are project-defined aspirational gates, not claims of external production certification.

The model is therefore presented as a measured portfolio/research model, not as a production-certified grammar checker.

---

Dataset and Training Setup

Final Conservative Training Dataset

The project uses:

• BEA-2019 W&I+LOCNESS grammar-error data;
• JFLEG multi-reference evaluation data;
• controlled synthetic grammar-error pairs;
• leakage-safe clean/identity examples;
• source-level split conflict resolution;
• dedicated clean-text over-correction evaluation.

The final audit reported zero source overlap between the training data and validation, test, and clean-test source sets.

---

Training Configuration

The selected model was fine-tuned locally on an NVIDIA RTX 5090.

---

Prompt Format

Training and inference use the same instruction-style prompt:

Task:
Correct the grammar, spelling, punctuation, and wording of the text while preserving the original meaning.

Rules:
- Preserve the original meaning.
- Do not add unsupported information.
- Do not remove important details.
- Keep named entities, numbers, technical terms, and product names unchanged unless clearly incorrect.
- Return only the corrected text.

Input:
{source_text}

Corrected Text:

Consistent prompting prevents a training/inference format mismatch.

---

Usage with Transformers

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

MODEL_ID = "anmol-unitmole/grammar-correction-flan-t5-base"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_ID)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

text = "She go to the office every day and complete her reports."

prompt = f"""Task:
Correct the grammar, spelling, punctuation, and wording of the text while preserving the original meaning.

Rules:
- Preserve the original meaning.
- Do not add unsupported information.
- Do not remove important details.
- Keep named entities, numbers, technical terms, and product names unchanged unless clearly incorrect.
- Return only the corrected text.

Input:
{text}

Corrected Text:"""

inputs = tokenizer(
    prompt,
    return_tensors="pt",
    truncation=True,
    max_length=384,
).to(device)

with torch.inference_mode():
    generated = model.generate(
        **inputs,
        max_new_tokens=192,
        num_beams=4,
        early_stopping=True,
    )

correction = tokenizer.decode(generated[0], skip_special_tokens=True)
print(correction)

---

ONNX Optimization Findings

The selected PyTorch model was exported to ONNX and dynamically INT8-quantized as an engineering experiment.

Artifact Size

100-Example Parity / Runtime Check

The parity test used a different single-example validation path from the batched evaluation latency shown earlier, so those latency figures should not be directly compared across tables.

Deployment decision: the INT8 ONNX experiment is retained as optimization evidence, but it is not claimed as the validated production/browser runtime because output parity and measured runtime were not strong enough.

---

What the Model Does Well

Measured strengths include:

• stronger correction quality than the Small FLAN-T5 experiments;
• high number preservation (99.72%);
• high technical-term preservation (99.63%);
• strong semantic similarity (93.84%);
• improved precision under conservative identity training;
• measurable reduction in unnecessary editing relative to Base 10%.

---

Known Limitations

• Clean-text over-correction remains substantial at 25.73%.
• Named-entity preservation is 93.57%, below the project's aspirational 98% gate.
• Exact sentence accuracy is intentionally strict and remains low.
• The model can miss valid corrections because conservative training trades some recall for precision.
• Multiple grammatical rewrites can be valid even when they differ from the benchmark reference.
• Long documents should be segmented rather than passed as one very long sequence.
• The INT8 ONNX experiment did not achieve sufficient parity/performance to be advertised as the validated live browser runtime.
• Human review is recommended for important professional text.

---

Intended Use

Suitable for:

• English grammar-correction experimentation;
• writing-assistance prototypes;
• sentence and paragraph correction;
• quality-report drafting support;
• customer-complaint narrative cleanup;
• root-cause and corrective-action writing assistance;
• professional communication demonstrations;
• encoder-decoder / FLAN-T5 portfolio work;
• GEC research and educational use.

---

Not Intended For

Do not use the model as the sole authority for:

• legal documents;
• medical documentation;
• financial or compliance-critical text;
• academic integrity decisions;
• safety-critical instructions;
• autonomous editing of confidential corporate text;
• any workflow where meaning or named entities must never change without human review.

---

Experimental Environment

Training and evaluation were executed locally using:

• GPU: NVIDIA GeForce RTX 5090
• VRAM: ~31.84 GB
• CUDA build: 13.0
• Python: 3.12.10
• PyTorch: 2.12.1 + CUDA 13.0
• Transformers: 4.57.6
• BF16: enabled
• TF32: enabled
• ERRANT: 3.0.2
• spaCy: 3.8.15

---

Reproducibility

The broader project contains reusable workflows for:

• BEA-2019 W&I+LOCNESS preparation;
• JFLEG preparation;
• controlled synthetic error generation;
• leakage auditing;
• token-length auditing;
• FLAN-T5-small/base fine-tuning;
• identity-ratio experiments;
• error-type balancing experiments;
• ERRANT evaluation;
• GLEU evaluation;
• semantic-similarity evaluation;
• preservation checks;
• clean over-correction benchmarking;
• latency benchmarking;
• ONNX export;
• INT8 quantization;
• ONNX parity validation;
• tracked edits;
• error categorization;
• batch correction;
• static portfolio deployment.

---

Responsible Use

Generated corrections can be incomplete, incorrect, overly aggressive, or meaning-changing.

Do not paste private, confidential, proprietary, customer, employee, or personally identifiable text into a public demonstration.

For important writing, review the original and corrected text before accepting changes.

---

License

This fine-tuned model is based on google/flan-t5-base, which is released under the Apache License 2.0. Users should also respect the licenses and usage terms of the datasets and other third-party resources used in the broader training/evaluation workflow.