prithivida-grammar-correcter-onnx-int8

prithivida/grammar_error_correcter_v1 — ONNX INT8 Quantized

This is an ONNX INT8 dynamically quantized version of prithivida/grammar_error_correcter_v1 for grammatical error correction in English.

This is a derivative work. All credit for the original model goes to Prithiviraj Damodaran (@prithivida). This repository only provides a quantized ONNX conversion for easier deployment. We do not claim ownership of the model architecture or weights.

Original Model

What’s in this repo?

• encoder_model.onnx — Encoder graph (INT8 quantized)
• decoder_model.onnx — Decoder graph (INT8 quantized)
• decoder_with_past_model.onnx — Decoder with KV-cache (INT8 quantized)
• config.json, generation_config.json — Model configuration
• tokenizer.json, tokenizer_config.json, special_tokens_map.json — Tokenizer files

Quantization Details

Benchmark Results — This Model

Benchmarked on a 50-sentence English grammar correction test set (2-thread CPU, Google Colab).

Quality

Speed & Size

Quantization Stability

Largest T5-base model. INT8 shows moderate quality drop exceeding the -2.0 BLEU threshold. Original model is part of the Gramformer library.

---

Cross-Model Comparison (7 Models Benchmarked)

This model was benchmarked alongside 6 other grammar correction models. All models were evaluated on the same 50-sentence test set under identical conditions.

PyTorch FP32 Baseline (all models)

ONNX INT8 Quantized (all models)

Statistical Significance (Bootstrap, 1000 resamples)

Key cross-model comparisons (INT8 variants):

Recommendation

---

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

model_id = "YOUR_USERNAME/prithivida-grammar-correcter-onnx-int8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

text = "gec: She go to school yesterday"
inputs = tokenizer([text], return_tensors="pt", max_length=128, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=128, num_beams=1, repetition_penalty=1.3)
corrected = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected)

Input prefix: gec:

Acknowledgments

All credit for the original model goes to Prithiviraj Damodaran (@prithivida). This repository only provides an ONNX INT8 quantized conversion to make the model easier to deploy in production environments (mobile, edge, browser).

Quantization and benchmarking performed as part of the Smart Desktop Keyboard Grammar Engine project.

Citation

If you use this model, please cite the original authors:

@misc{prithivida_onnx_int8,
  title = {prithivida/grammar_error_correcter_v1 — ONNX INT8 Quantized},
  note = {Quantized version of prithivida/grammar_error_correcter_v1},
  url = {https://huggingface.co/prithivida/grammar_error_correcter_v1},
}