vennify-t5-base-grammar-correction-onnx-int8

vennify/t5-base-grammar-correction — ONNX INT8 Quantized

License Notice: This model inherits the CC BY-NC-SA 4.0 license from the original. It may only be used for non-commercial purposes.

This is an ONNX INT8 dynamically quantized version of vennify/t5-base-grammar-correction for grammatical error correction in English.

This is a derivative work. All credit for the original model goes to Vennify (Eric Fillion). This repository only provides a quantized ONNX conversion for easier deployment. We do not claim ownership of the model architecture or weights.

Original Model

What’s in this repo?

• encoder_model.onnx — Encoder graph (INT8 quantized)
• decoder_model.onnx — Decoder graph (INT8 quantized)
• decoder_with_past_model.onnx — Decoder with KV-cache (INT8 quantized)
• config.json, generation_config.json — Model configuration
• tokenizer.json, tokenizer_config.json, special_tokens_map.json — Tokenizer files

Quantization Details

Benchmark Results — This Model

Benchmarked on a 50-sentence English grammar correction test set (2-thread CPU, Google Colab).

Quality

Speed & Size

Quantization Stability

Highest baseline quality but large size (426 MB INT8). INT8 introduced some language-switching artifacts on edge cases.

---

Cross-Model Comparison (7 Models Benchmarked)

This model was benchmarked alongside 6 other grammar correction models. All models were evaluated on the same 50-sentence test set under identical conditions.

PyTorch FP32 Baseline (all models)

ONNX INT8 Quantized (all models)

Statistical Significance (Bootstrap, 1000 resamples)

Key cross-model comparisons (INT8 variants):

Recommendation

---

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

model_id = "YOUR_USERNAME/vennify-t5-base-grammar-correction-onnx-int8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

text = "grammar: She go to school yesterday"
inputs = tokenizer([text], return_tensors="pt", max_length=128, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=128, num_beams=1, repetition_penalty=1.3)
corrected = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected)

Input prefix: grammar:

Acknowledgments

All credit for the original model goes to Vennify (Eric Fillion). This repository only provides an ONNX INT8 quantized conversion to make the model easier to deploy in production environments (mobile, edge, browser).

Quantization and benchmarking performed as part of the Smart Desktop Keyboard Grammar Engine project.

Citation

If you use this model, please cite the original authors:

@misc{vennify_t5_base_onnx_int8,
  title = {vennify/t5-base-grammar-correction — ONNX INT8 Quantized},
  note = {Quantized version of vennify/t5-base-grammar-correction},
  url = {https://huggingface.co/vennify/t5-base-grammar-correction},
}