visheratin-t5-tiny-grammar-correction-onnx-int8

visheratin/t5-efficient-tiny-grammar-correction — ONNX INT8 Quantized

This is an ONNX INT8 dynamically quantized version of visheratin/t5-efficient-tiny-grammar-correction for grammatical error correction in English.

This is a derivative work. All credit for the original model goes to Alexander Visheratin (@visheratin). This repository only provides a quantized ONNX conversion for easier deployment. We do not claim ownership of the model architecture or weights.

Original Model

What’s in this repo?

• encoder_model.onnx — Encoder graph (INT8 quantized)
• decoder_model.onnx — Decoder graph (INT8 quantized)
• decoder_with_past_model.onnx — Decoder with KV-cache (INT8 quantized)
• config.json, generation_config.json — Model configuration
• tokenizer.json, tokenizer_config.json, special_tokens_map.json — Tokenizer files

Quantization Details

Benchmark Results — This Model

Benchmarked on a 50-sentence English grammar correction test set (2-thread CPU, Google Colab).

Quality

Speed & Size

Quantization Stability

Smallest and fastest model. Ideal for mobile/edge deployment where size is critical. Statistically identical quality to visheratin-mini (p=0.824).

---

Cross-Model Comparison (7 Models Benchmarked)

This model was benchmarked alongside 6 other grammar correction models. All models were evaluated on the same 50-sentence test set under identical conditions.

PyTorch FP32 Baseline (all models)

ONNX INT8 Quantized (all models)

Statistical Significance (Bootstrap, 1000 resamples)

Key cross-model comparisons (INT8 variants):

Recommendation

---

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

model_id = "YOUR_USERNAME/visheratin-t5-tiny-grammar-correction-onnx-int8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

text = "(none — empty string)She go to school yesterday"
inputs = tokenizer([text], return_tensors="pt", max_length=128, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=128, num_beams=1, repetition_penalty=1.3)
corrected = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected)

Input prefix: (none — empty string)

Acknowledgments

All credit for the original model goes to Alexander Visheratin (@visheratin). This repository only provides an ONNX INT8 quantized conversion to make the model easier to deploy in production environments (mobile, edge, browser).

Quantization and benchmarking performed as part of the Smart Desktop Keyboard Grammar Engine project.

Citation

If you use this model, please cite the original authors:

@misc{visheratin_tiny_onnx_int8,
  title = {visheratin/t5-efficient-tiny-grammar-correction — ONNX INT8 Quantized},
  note = {Quantized version of visheratin/t5-efficient-tiny-grammar-correction},
  url = {https://huggingface.co/visheratin/t5-efficient-tiny-grammar-correction},
}