ModernBERT-base-ONNX-Attentions

ModernBERT-base ONNX (with Attention Weights & INT8 Quantization)

This is an optimized version of the ModernBERT-base model, exported to ONNX with explicit support for attention weights and INT8 dynamic quantization.

Unique Features

• Output Attentions: Unlike standard ONNX exports, this version has the output_attentions flag enabled in the graph. You can access matrices for all 22 layers (including Global and Local attention).
• 8k Context Support: Fully supports the 8192 token sequence length of the original ModernBERT.
• INT8 Quantized: The model_quantized.onnx file is 4x smaller (~150MB) and significantly faster on CPUs with AVX-512/VNNI support.
• Multilingual Support: Excellent performance on both English and Russian technical texts.

Files included

• model.onnx: Full FP32 model (supports attention).
• model_quantized.onnx: INT8 Quantized model (supports attention).
• tokenizer.json: Standard ModernBERT tokenizer.
• config.json & ort_config.json: Configuration files.

Usage (Python + ONNX Runtime)

import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

# Load quantized model
session = ort.InferenceSession("model_quantized.onnx")
tokenizer = AutoTokenizer.from_pretrained(".")

inputs = tokenizer("Your text here", return_tensors="np")
# Request only the last layer of attention for efficiency
outputs = session.run(["attentions.21"], dict(inputs))

# attentions.21 shape: [batch, heads, seq, seq]
last_layer_attn = outputs[0]
print(last_layer_attn.shape)

Purpose

Ideal for:

• Context Pruning: Intelligent removal of "noise" from long logs based on attention focus.
• Explainable AI: Visualizing word dependencies in long technical contexts.
• Edge Inference: Running high-performance BERT analysis without PyTorch/GPU.

License

Apache 2.0 (same as base model).

---

Created using jaagent context optimization tools.