sage-fredt5-onnx

SAGE FRED-T5 — ONNX (INT8)

ONNX INT8-quantized version of ai-forever/FRED-T5-large-spell (SAGE spelling correction) for on-device Russian text correction in macOS apps.

Model Details

• Architecture: T5ForConditionalGeneration (FRED-T5-large fine-tuned for spelling)
• Format: ONNX INT8 quantized, encoder-decoder with KV-cache
• Size: ~488 MB total (INT8), ~1.0 GB (FP32)
• Input: Tokenized Russian text (BPE, vocab size 50364)
• Output: Corrected Russian text (seq2seq generation)
• Task prefix: "Исправьте: " prepended to input text

Model Files

INT8 (recommended for on-device)

• int8/encoder_model_int8.onnx — Encoder (43 MB)
• int8/decoder_with_past_model_int8.onnx — Decoder with KV-cache (71 MB)
• int8/decoder_model_int8.onnx — Decoder without cache (74 MB)
• int8/decoder_model_merged_int8.onnx — Merged decoder (293 MB)
• int8/vocab.json, int8/merges.txt — BPE tokenizer
• int8/config.json, int8/tokenizer.json, etc.

FP32 (full precision)

• fp32/encoder_model.onnx — Encoder (171 MB)
• fp32/decoder_with_past_model.onnx — Decoder with KV-cache (281 MB)
• fp32/decoder_model.onnx — Decoder without cache (293 MB)

Usage

On-device Russian text correction of speech transcripts. Runs via ONNX Runtime C API with greedy decoding.

// Swift — Seq2Seq inference
let engine = Seq2SeqOnnxEngine()
try engine.loadModels(from: "/path/to/sage-fredt5/int8/")
let corrected = try engine.generate(inputIDs: tokenizer.encode("Исправьте: " + text))

Attribution

Base model FRED-T5-large and spelling fine-tune FRED-T5-large-spell by AI Forever / Sber AI. Part of the SAGE spelling correction framework. ONNX conversion and INT8 quantization by @smkrv.