laya-multilingual-onnx

Laya Multilingual — ONNX for JavaScript / TypeScript

ONNX export of convaiinnovations/laya-multilingual, retaining its custom typed-decision head. It routes and classifies a supplied state using request-defined options in 100+ languages.

This is a Node.js package example. The 322M-parameter model is roughly 1.3 GB in FP32, so it is generally unsuitable for ordinary browser delivery. Use it behind a service, in Electron, or in an appropriately provisioned Node.js process.

What is included

• model.onnx — encoder plus Laya choice/score/noul and act/escalate heads.
• tokenizer/ — original mmBERT tokenizer.
• src/index.ts — small ONNX Runtime Node wrapper that exactly reconstructs the Laya request sequence.
• examples/basic.ts — general multilingual routing example.
• examples/safety-gate.ts — conservative AI-agent tool safety gate.

Install and run

git clone https://huggingface.co/Sharjeelbaig/laya-multilingual-onnx
cd laya-multilingual-onnx
npm install
npm run example

Or import the wrapper into an existing Node.js TypeScript project:

npm install onnxruntime-node @huggingface/tokenizers

import { LayaOnnx } from './src/index.js';

const model = await LayaOnnx.fromDirectory('/models/laya-multilingual-onnx');
const answer = await model.predict(
  { body: 'I was charged twice for invoice 4411. Please refund me.' },
  {
    type: 'choice',
    instructions: 'Which team should handle body?',
    criteria: {
      billing: 'invoices, payments, and refunds',
      technical: 'bugs and outages',
      sales: 'pricing and purchase questions',
    },
  },
);
console.log(answer.choice, answer.probabilities, answer.confidence);

choice accepts named options. noul is a two-way boolean question (false / true); score accepts ordered criteria. Keep choice questions to roughly 20 options or fewer.

AI agent safety gate

The included examples/safety-gate.ts gates send_email, issue_refund, and delete_record tool calls. It only allows an action when the model selects allow, its entropy-based confidence clears the local threshold, and its separate decision head does not request escalation. Everything else goes to human review; block is denied.

npm run safety-gate

Adapt the options and threshold to your policy and validate them on representative, labeled data before deployment. This checkpoint ships uncalibrated and must not be the sole control for irreversible, regulated, or high-impact actions. Pair it with deterministic authorization checks, audit logs, rate limits, and human review.

Inputs and outputs

predict(state, question) serializes an object or array as JSON (or accepts a string) and returns:

{
  choice: 'billing',
  probabilities: { billing: 0.82, technical: 0.12, sales: 0.06 },
  confidence: 0.44,             // entropy-based; not calibrated probability of correctness
  act: 'act',
  actProbabilities: { act: 0.91, escalate: 0.09 }
}

The output classes are defined by each question—not fixed labels baked into the model.

Export provenance and verification

Exported from the Apache-2.0 checkpoint with PyTorch torch.onnx.export, opset 17. The export uses fixed maximum input dimensions [1, 1024] and up to 20 option markers, matching the wrapper. Before release, option logits and act logits were compared with the original PyTorch inference graph (maximum absolute differences: 2.6e-6 and 9.8e-4, respectively).

Original model: convaiinnovations/laya-multilingual. Model behavior, benchmark claims, and limitations originate with Convai Innovations.