onca-2.0-12b-INT8

<p align="center"> <img src="./assets/onca-logo-horizontal.svg" alt="Onca 2.0 logo" width="520"> </p>

ONCA 2.0 8-bit

<p align="center"> <img src="./assets/onca20-holdout-primary-comparison.png" alt="ONCA 2.0 four-task primary-holdout comparison" width="900"> </p> <p align="center">_{BF16 reference scores on the unchanged 1,309-example primary holdout; compare values within each task panel.}</p>

Summary

ONCA 2.0 is an open oncology language model for trial screening, clinical reasoning, pathology extraction, and variant evidence interpretation. It builds on google/gemma-4-12B-it with continued supervised fine-tuning on a provenance-labeled oncology corpus while retaining the four-task ONCA 1.5 evaluation contract.

This repository contains the merged 8-bit BitsAndBytes checkpoint, quantized from the BF16 reference release and exported on 2026-06-14. It provides a lower-memory Transformers deployment for pancreatic cancer and oncology-adjacent research workflows.

At a Glance

The INT8 export passed loading checks but was not independently rebenchmarked across the full holdout; small quantization-related differences may occur.

Quick Start

Use recent versions of Transformers, Accelerate, and BitsAndBytes with Gemma 4 support.

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "Joesh1/onca-2.0-12B-INT8"

processor = AutoProcessor.from_pretrained(model_id)
tokenizer = processor.tokenizer
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

# Example: criterion-aware trial screening with structured output
messages = [{
    "role": "user",
    "content": (
        "Patient: metastatic pancreatic adenocarcinoma; ECOG 1; "
        "no prior metastatic-line therapy. Trial: metastatic PDAC, ECOG 0-1, "
        "no prior metastatic-line therapy. Return JSON with keys eligible, "
        "reason, and missing_information."
    ),
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=160, do_sample=False)
answer = tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(answer)

The saved config.json contains the INT8 quantization metadata. For structured workflows, request exact fields, provide all relevant criteria, ask for explicit uncertainty, and prefer deterministic decoding.

Training Scope

The source BF16 checkpoint was trained on 25,302 examples. Validation, test, and primary-holdout sets retain the ONCA 1.5 four-task evaluation contract.

Related Releases

• onca-2.0-12B: BF16 reference release.
• onca-2.0-12B-INT8: 8-bit BitsAndBytes release (this page).
• onca-2.0-12B-INT4: 4-bit BitsAndBytes release.
• onca-2.0-12B-GGUF: llama.cpp-compatible GGUF collection.

Limitations

• This is a research model, not a clinical decision system.
• Outputs require review by qualified experts before real-world use.
• Structured or parser-valid output does not guarantee factual correctness.
• Quantization may cause behavior to differ slightly from the BF16 reference checkpoint.
• The full primary holdout was not independently rerun for this INT8 export.

Citation

A formal ONCA 2.0 citation will be added with the accompanying manuscript. Until then, cite this model repository and the exact version used.

Acknowledgements

ONCA 2.0 continues the ONCA project lineage and builds on Google Gemma and the open-data contributors whose datasets supported training and evaluation.