gec-t5-small-coedit-onnx-int8

Unbabel/gec-t5_small — ONNX INT8 Quantized

This is an ONNX INT8 dynamically quantized version of Unbabel/gec-t5_small for grammatical error correction in English.

This is a derivative work. All credit for the original model goes to Unbabel. This repository only provides a quantized ONNX conversion for easier deployment. We do not claim ownership of the model architecture or weights.

Original Model

What’s in this repo?

• encoder_model.onnx — Encoder graph (INT8 quantized)
• decoder_model.onnx — Decoder graph (INT8 quantized)
• decoder_with_past_model.onnx — Decoder with KV-cache (INT8 quantized)
• config.json, generation_config.json — Model configuration
• tokenizer.json, tokenizer_config.json, special_tokens_map.json — Tokenizer files

Quantization Details

Benchmark Results — This Model

Benchmarked on a 50-sentence English grammar correction test set (2-thread CPU, Google Colab).

Quality

Speed & Size

Quantization Stability

Recommended model. Best quality-to-size ratio. Rock-solid INT8 stability with only 1 sentence regression. Based on the paper 'A Simple Recipe for Multilingual Grammatical Error Correction'.

---

Cross-Model Comparison (7 Models Benchmarked)

This model was benchmarked alongside 6 other grammar correction models. All models were evaluated on the same 50-sentence test set under identical conditions.

PyTorch FP32 Baseline (all models)

ONNX INT8 Quantized (all models)

Statistical Significance (Bootstrap, 1000 resamples)

Key cross-model comparisons (INT8 variants):

Recommendation

---

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

model_id = "YOUR_USERNAME/gec-t5-small-coedit-onnx-int8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

text = "gec: She go to school yesterday"
inputs = tokenizer([text], return_tensors="pt", max_length=128, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=128, num_beams=1, repetition_penalty=1.3)
corrected = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected)

Input prefix: gec:

Acknowledgments

All credit for the original model goes to Unbabel. This repository only provides an ONNX INT8 quantized conversion to make the model easier to deploy in production environments (mobile, edge, browser).

Quantization and benchmarking performed as part of the Smart Desktop Keyboard Grammar Engine project.

Citation

If you use this model, please cite the original authors:

@misc{coedit_small_onnx_int8,
  title = {Unbabel/gec-t5_small — ONNX INT8 Quantized},
  note = {Quantized version of Unbabel/gec-t5_small},
  url = {https://huggingface.co/Unbabel/gec-t5_small},
}