OpenMed-mLiteClinical-IrishPPSN-135M-v1-fp16

OpenMed-mLiteClinical-IrishPPSN-135M-v1-fp16

OpenMed-mLiteClinical-IrishPPSN-135M-v1-fp16 is a half-precision GPU-oriented release of temsa/OpenMed-mLiteClinical-IrishPPSN-135M-v1.

Use this release when you want the same PPSN masking behavior as the full model with lower memory use and faster inference on ROCm / CUDA GPUs, or on CPU when your serving path is low-batch and short-sequence.

What This Release Is

• A standard transformers checkpoint stored in fp16
• Derived from temsa/OpenMed-mLiteClinical-IrishPPSN-135M-v1
• Intended for PPSN masking with the bundled word_aligned decoder
• Works on CPU and GPU; it showed the largest speedup on GPU, and a CPU speedup on the current low-batch word_aligned path

Recommended Inference

Use the bundled entrypoint, which loads the checkpoint with dtype=auto so the stored fp16 weights are used directly. This is a good fit for low-batch CPU serving and for GPU inference:

python3 inference_word_aligned.py --ppsn-min-score 0.4 --text "My PPSN is 1234567TW and I need help with my housing grant." --json

To load directly from the Hub:

python3 inference_word_aligned.py --model temsa/OpenMed-mLiteClinical-IrishPPSN-135M-v1-fp16 --ppsn-min-score 0.4 --text "My PPSN is 1234567TW and I need help with my housing grant." --json

Benchmark Summary

Measured on the multilingual PPSN suite spanning gov data, citizen-to-government chat, and HSE medical text in English, Irish Gaelic, and additional European / Ukrainian / Russian / Chinese / Japanese examples.

CPU behavior depends on workload shape. The end-to-end word_aligned benchmark below is favorable to fp16, but the batched CPU runtime matrix in eval/runtime_matrix_cpu_fp32_vs_fp16_compact.md shows that fp16 loses to fp32 once batch size and sequence length grow.

For CPU guidance in this model card:

• short text: about <= 32 tokenizer tokens
• 33-63 tokens: gray zone, benchmark your workload
• >= 64 tokens: not short for this recommendation
• low batch: batch_size = 1
• batch_size = 2-4: gray zone, benchmark your workload
• batch_size >= 8: not low batch for this recommendation

Practical CPU rule:

• Prefer this fp16 repo for batch_size = 1 and about <= 32 tokens
• Prefer the canonical fp32 repo once batch size or sequence length grows materially

Small PPSN regression suites with fp16 matched the full model in this workspace:

• User raw F1: 0.8000
• QA v6 validated F1: 0.6667
• QA v8 F1: 0.7385

Tradeoff

• Roughly half the model size vs the fp32 checkpoint
• Same measured PPSN quality as the fp32 release in these tests
• Faster GPU inference on the AMD ROCm setup used here
• Faster CPU inference than the fp32 checkpoint on the current low-batch end-to-end path
• In a batched CPU forward matrix, fp16 became slower than fp32 once batch size and sequence length increased; see eval/runtime_matrix_cpu_fp32_vs_fp16_compact.md
• The int8 release is still the highest-throughput CPU option, but it gives up more PPSN quality

Included Files

• Core model:
  • model.safetensors
  • config.json
  • precision.json
  • tokenizer.json
  • tokenizer_config.json
  • special_tokens_map.json
  • label_meta.json
  • vocab.txt
• Inference / QA:
  • inference_word_aligned.py
  • qa_config.json
  • pyproject.toml
• Evaluation:
  • eval/

License

This reduced-precision derivative is distributed under Apache-2.0, consistent with the canonical full model and upstream OpenMed base model. See NOTICE for attribution.

<!-- portfolio-comparison:start -->

Portfolio Comparison

Updated: 2026-03-16.

Use this section for the fastest public comparison across the temsa PII masking portfolio.

• The first core table only includes public checkpoints that ship both comparable q8 accuracy and q8 CPU throughput.
• The first PPSN table only includes public artifacts that ship comparable PPSN accuracy and CPU throughput.
• Missing cells in the archive tables mean the older release did not ship that metric in its public bundle.
• DiffMask rows use the reconciled clean_single_pass harness that matches the deployed runtime.
• GlobalPointer rows use the public raw-only span-matrix release bundle and its packaged q8 ONNX artifact.
• The same content is shipped as PORTFOLIO_COMPARISON.md inside each public model repo.

Irish Core PII: Comparable Public Checkpoints

Irish Core PII: Other Public Checkpoints

Finance-boundary q8 F1 is 1.0000 for OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc6, OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc7, OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8, and all public IrishCore-DiffMask releases from rc1 to rc6. OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc5 ships 0.8750 on that public q8 suite.

PPSN-Only: Comparable Public Artifacts

PPSN-Only: Historical Public Checkpoints

If you need the strongest current raw-only Irish core model, start with IrishCore-GlobalPointer-135M-v1-rc4. If you need the fastest CPU-first raw-only line, compare it against IrishCore-DiffMask-135M-v1-rc6. If you need a PPSN-only artifact, compare the canonical fp32, fp16, and q8 variants of OpenMed-mLiteClinical-IrishPPSN-135M-v1 directly in the table above.

<!-- portfolio-comparison:end -->