MedPsy-4B-ONNX

MedPsy-4B - ONNX 4-bit (GPTQ)

An ONNX conversion of qvac/MedPsy-4B for ONNX Runtime, quantized to a 4-bit body with GPTQ and 8-bit sensitive layers, for on-device use on Android, desktop and the browser (Transformers.js). The model is QVAC's MedPsy medical fine-tune of Qwen3-4B; OpenMed made and published this conversion and is not affiliated with or endorsed by the model's authors.

Revision of 2026-09-25. The weights are now GPTQ-calibrated. They replace the round-to-nearest build first published on 2026-09-23, at the same size and in the same file layout; mean next-token divergence from the source fell by 36 to 53% on three held-out texts (table below).

Size and fidelity

Measured against the source model (Transformers, FP32) with ONNX Runtime 1.30.0 on CPU, over 4,092 scored tokens per text; none of these texts is in the calibration set:

The width is higher than the "4-bit" label suggests because the tensors that lose most at 4 bits, and the shared embedding table, are kept at 8 bits. Weights exclude the rotary-position tables stored in the graph.

Quantization

The tokenizer, chat template and generation defaults are the upstream files, unchanged. Exact settings, tool versions, per-corpus results and every file's SHA-256 are in openmed_build.json; the graph's metadata_props record the source revision and the modification notice.

Architecture

Quick start

Transformers.js (browser or Node)

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline("text-generation", "OpenMed/MedPsy-4B-ONNX", { dtype: "q4" });
const messages = [
  { role: "user", content: "Explain the difference between a panic attack and a heart attack in plain language." },
];
const output = await generator(messages, { max_new_tokens: 256, do_sample: false });
console.log(output[0].generated_text.at(-1).content);

ONNX Runtime (Python, Android and elsewhere)

The graph is a standard ONNX Runtime decoder: feed input_ids, attention_mask and the past_key_values.* cache inputs, then feed each present* output back as the matching past* input on the next step. It uses ONNX Runtime's com.microsoft operators (MatMulNBits, GatherBlockQuantized, GroupQueryAttention), so run it with ONNX Runtime 1.30 or later (onnxruntime-android on Android).

Tested with onnxruntime 1.30.0 (CPU execution provider) on macOS and with Transformers.js 4.3.0, which produced identical greedy tokens. It has not yet been measured on an Android device.

File set

Intended use

For research and education. QVAC describes MedPsy as a model for research and educational purposes, trained on data generated from CC-BY-NC datasets; see the upstream card. It is not a medical device and not a substitute for professional medical advice, diagnosis or treatment. Outputs can be wrong; verify them.

Licence

Distributed under the Apache License 2.0, the source model's licence (with the upstream attributions in ATTRIBUTIONS.md). The ONNX graphs and weight files in onnx/ were converted and quantized from the source model's PyTorch weights by OpenMed; all other files are unchanged upstream copies.

Source: qvac/MedPsy-4B at revision 77aec3ab8c6eae92e18951e0a86c68c84a067d28. Please cite the original model when you use this conversion.