laya-onnx

Laya — ONNX (fp32, dynamic shapes)

An ONNX export of convaiinnovations/laya, the calibrated typed-decision model (ModernBERT-large encoder, 421M parameters). Laya answers choice, score and noul (calibrated yes/no) questions about a text or JSON state in one forward pass.

This export keeps batch, sequence length and option count dynamic. The existing community export bakes the sequence length into a Reshape inside ModernBERT's attention, so it only runs at 512 tokens. Here you can use it at any length up to 1,024 tokens (the export's bound; Laya itself uses 512), on any ONNX Runtime backend. It's also the starting point for running Laya on the Snapdragon X NPU: piffie/laya-snapdragon downloads it, pins a few fixed-shape buckets and compiles them for the Qualcomm HTP (17 ms per question vs 97 ms on the CPU, same answers).

Files

Inputs: input_ids [batch, seq] int64, attention_mask [batch, seq] int64, marker_pos [batch, markers] int64, marker_mask [batch, markers] bool, qtype [batch] int64 (0 choice, 1 score, 2 noul). Outputs: logits [batch, markers] (one per option, before temperature calibration) and act [batch, 2].

The raw logits are only half the model. You also need Laya's prompt layout (question, option markers, state) and its temperature calibration. Use them from laya-snapdragon (laya_snapdragon.Agent, which also runs this file on the CPU with device="cpu") or from laya-mlx. Both are torch-free and match upstream.

Provenance

• Weights: convaiinnovations/laya at revision 1c5edc17a7acd8701df6fc341c0d179f1c62c982
• Model code: NandhaKishorM/laya at 6a5819129eb220570792e417e49723d697efd76f
• Export: torch.onnx.export(dynamo=True, dynamic_shapes=…, opset_version=18, optimize=True) with torch 2.14 (CPU) on native Windows ARM64. The script is laya_snapdragon/build.py (export) in the GitHub repo.
• Nothing is quantized, pruned or retrained.

Fidelity

Against upstream PyTorch Laya (CPU), on laya-mlx's parity fixtures: 16 cases, 63 questions, 8 languages, up to 512 tokens and 20 options, with identical input tensors:

License

Apache-2.0, like the upstream model and code. Laya and its weights are by Convai Innovations. This repo only re-packages them as ONNX.