timesfm-3.0-onnx

TimesFM 3.0 ONNX

ONNX conversion and CPU quantization experiments for google/timesfm-3.0-pytorch.

This repository focuses on running TimesFM 3.0 efficiently on CPU using ONNX Runtime, including FP32 reference inference and lower-precision INT8 / INT4 variants.

Models

FP32 ONNX

Reference ONNX model with near-identical output to the original PyTorch model.

• Model size: ~1.27 GB
• Context length: 128
• Forecast horizon: 64
• Output quantiles: 9
• Recommended as the accuracy reference

Observed PyTorch vs FP32 ONNX numerical difference:

• Max absolute error: ~2.15e-6
• Mean absolute error: ~5.74e-7

INT8 ONNX

Experimental CPU-optimized INT8 model.

• Model size: ~323 MB
• Approximately 4x smaller than FP32
• Faster CPU inference
• Dynamic / weight quantization

Current INT8 quantization significantly affects forecasting accuracy and should be treated as experimental.

INT4 ONNX

Experimental weight-only INT4 model using ONNX Runtime MatMulNBits.

• Designed for reduced model size
• Uses 4-bit block-wise weight quantization
• CPU-oriented inference
• Experimental quality and runtime characteristics

INT4 quantization may introduce larger forecast degradation than INT8 depending on the quantized layers and CPU implementation.

Files

Typical files in this repository:

timesfm3-fp32-c128-h64.onnx
timesfm3-fp32-c128-h64.onnx.data

timesfm3-int8-c128-h64.onnx
timesfm3-int8-full-c128-h64.onnx

timesfm3-int4-full-c128-h64.onnx

onnx_config.json

quantization_report.json
int8_full_benchmark.json
int4_full_benchmark.json