serialize-parity-v1

ferrotorch/serialize-parity-v1

Phase G.3 of ferrotorch's real-artifact-driven development (#1169). Pins canonical references for ferrotorch-serialize's four format loaders/exporters so the rust crate's parsers and emitters can be verified byte-exact against the upstream toolchains they target.

Targets

• .pth load — resnet18-pth/resnet18-5c106cde.pth is the official torchvision checkpoint (https://download.pytorch.org/models/resnet18-5c106cde.pth). reference_state_dict/<key>.bin carries each tensor as [u32 ndim][u32 shape...][f32 bytes]. The rust harness dumps the same per-tensor binaries via ferrotorch_serialize::load_pytorch_state_dict and compares byte-exact (max_abs = 0).

• SafeTensors round-trip — safetensors-rt/resnet18.safetensors is the same resnet18 state_dict re-saved via safetensors.torch.save_file. References are the same per- tensor binaries as the .pth target. The rust harness compares byte-exact (max_abs = 0).

• GGUF load — gguf/SmolLM-135M-Instruct-Q4_K_M.gguf is the upstream unsloth/SmolLM-135M-Instruct-GGUF checkpoint. reference_dequant/<name>.bin carries dequantized f32 tensors for a deterministic stride-sampled subset of layers, produced by python's gguf.GGUFReader. The rust harness reproduces those under max_abs <= 1e-4 (Q4_K group scaling has a known noise floor between implementations).

• ONNX export — onnx-mlp/ carries:

  • mlp_weights.bin — fixed-seed (torch.manual_seed(42)) weights for a Linear(4 -> 8) + ReLU + Linear(8 -> 2) MLP. The rust side reads these so its in-memory MLP matches torch's bit-for-bit before export.
  • input_{zeros,ones,random}.bin — three fixed inputs.
  • torch_forward_{zeros,ones,random}.bin — reference forward outputs from torch.nn.Sequential.

  The rust harness builds the same MLP from mlp_weights.bin, exports it via ferrotorch_serialize::export_onnx, dumps the rust-side ferrotorch forward, and the python verifier loads the rust-emitted .onnx via onnxruntime.InferenceSession and asserts cosine_sim >= 0.9999 + max_abs <= 1e-5 across (rust-onnx vs rust-ferrotorch) AND (rust-onnx vs torch).

Provenance

• Pin script: scripts/pin_pretrained_serialize_fixtures.py.
• Verifier: scripts/verify_serialize_inference.py.
• Rust dumps: ferrotorch-serialize/examples/serialize_{pth,safetensors,gguf,onnx_export}_dump.rs.
• Cargo test wrapper: ferrotorch-serialize/tests/conformance_format_parity.rs.
• Tracking issue: https://github.com/dollspace-gay/ferrotorch/issues/1169.
• SHA-256 of bundle.tar (pinned in ferrotorch-hub/src/registry.rs): 7c20267db5706421e7367c4d275346114a43ff6d55e6ff1aa11069bc45562296.

Upstream licenses

• resnet18 weights — BSD-3-Clause (torchvision).
• SmolLM2-135M-Instruct-GGUF — Apache-2.0 (upstream unsloth mirror of HuggingFace's HuggingFaceTB/SmolLM2-135M-Instruct).
• ferrotorch fixtures themselves — Apache-2.0 / MIT.