assay-transfer-tool

assay-transfer-tool

A full-parameter SFT of the Qwen3ForCausalLM Intern-S1-mini language backbone (jiosephlee/Intern-S1-mini-lm) for binary assay-transfer prediction on small molecules. Given a molecule (SMILES) and a paired-choice prompt, the model answers with (A) or (B).

The model uses the Intern SMILES-aware tokenizer, so load with trust_remote_code=True.

Training

• Base model: jiosephlee/Intern-S1-mini-lm (Qwen3ForCausalLM, vocab 153216)
• Dataset: jiosephlee/assay-transfer-intern (158,429 train rows)
• Regime: full-parameter BF16, packing + padding-free, gradient checkpointing, Liger fused linear cross-entropy, paged_adamw_8bit
• Chat template: enabled, thinking disabled; completions are (A) / (B)
• LR: 2e-5 base, with SMILES input-embedding rows trained at 1.5x
• Max length: 4096, 1 epoch

This checkpoint is the best-validation snapshot (step 50), promoted by the assay-transfer callback on binary macro-F1.

Validation metrics (source_value split, n=1703)

Per-assay macro-F1: Fa 0.532, Fg 0.593, Fh 0.704, oral_bioavailability 0.600, oral_exposure 0.732. Per-Tanimoto-bucket macro-F1: high 0.664, low 0.600.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("jiosephlee/assay-transfer-tool", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("jiosephlee/assay-transfer-tool", torch_dtype="bfloat16")