LFM2.5-1.2B-Instruct-ONNX-Vulkan

LFM2.5-1.2B-Instruct ONNX - Vulkan / dual-role decoders

Vulkan-ready Instruct decoders for the LFM2.5 Audio + Instruct dual-role bundle (role-isolated sessions; the Instruct role handles text/tool I/O while the audio role handles speech). This repo holds only the modified Instruct artifacts - the audio q4 stack (decoder, encoder, detokenizer, vocoder, embeddings) is byte-identical to the official LiquidAI/LFM2.5-Audio-1.5B-ONNX release and is referenced from there, not re-hosted.

Files

Graph contract

Both decoders expose the same 24-input contract as the audio q4 decoder: inputs_embeds (f32, first), attention_mask (i64), and per-layer past_conv.* / past_key_values..{key,value} (f32). Outputs: logits + present_conv./ present.*.{key,value} (the q4 graph has no hidden_states output - the Instruct role reads logits only).

What was modified (vs the official Liquid Instruct ONNX release)

Source: LiquidAI/LFM2.5-1.2B-Instruct-ONNX @ commit 10f72e70abf67ac0fd7ebf15bc5854726891d864 (model_q4.onnx). The official release was NOT runnable as-is in the target runtime (onnx-vulkan-rs, a Vulkan ONNX interpreter); three fixes were applied - see instruct_q4_provenance.json for the full record:

1. Embedding contract: the release takes input_ids (i64) and embeds internally via a custom q4 GatherBlockQuantized over a q4-quantized embed table. That node was removed and the input swapped to inputs_embeds (f32) - the runtime feeds embeddings from embed_tokens.bin (byte-identical to the audio q4 24-input contract). The weight-tied q4 lm_head (which reuses the embedding's scales + zero points) is preserved, so logits stay q4-quantized.
2. Node order: the Liquid export node list was non-topological (110 violations); it was stably topo-sorted (ties on original index; 0 violations after).
3. External-data locations repointed to decoder_q4.onnx_data.

The f32 fallback decoder derives from the same export family's fp16 graph (topo-sorted) with all 148 weight initializers swapped from the Instruct bf16 checkpoint (LiquidAI/LFM2.5-1.2B-Instruct @ 6314d2b7cf28a6ae9de9d3e77dcfcd9c9f281c77) and an exact f16->f32 upcast of the remaining 150 f16 initializers (the target engine is f32/i32/q4-only; it has no f16 compute path).

Quantization format note (q4)

The Instruct q4 export is asymmetric MatMulNBits: 93 nodes, bits=4, block_size=32, with a 4th input - a per-block packed-int4 zero point (uint8 tensor of shape [n][k/64], two int4 per byte, low nibble first). The Audio release is the 3-input symmetric form (zero point == 8). A runtime must implement the asymmetric form to use decoder_q4.onnx; the onnx-vulkan-rs fork that produced this bundle adds packed-int4 zero-point variants of its Q4 kernels for exactly this (fork: https://github.com/matbeedotcom/onnx-vulkan-rs-edge, commit 2d660f2). onnxruntime CPU runs the 4-input form natively.

Validation

• CPU (onnxruntime, greedy): the B4 tool-call prompt (88 ids) produces 21/21 tokens bit-identical to the bf16 PyTorch reference, including the tool_call_start / tool_call_end wrapper tokens: [get_weather(city="Paris")]Checking the current weather in Paris.
• Reproducibility: the committed export pipeline (dual_role/instruct_q4_export.py in the project repo) re-derives decoder_q4.onnx byte-identically (sha256 d839af62...).

License

LFM Open License v1.0 - inherited from the upstream Liquid releases (see LICENSE). The modifications are graph surgery + exact dtype conversion only; no weight values were retrained or requantized.