pgl-cancer-0.6B

Model Details

• Base Model: Qwen3-0.6B (32K context window)
• Model Type: Causal Language Model (text-generation)
• Training Context: Trained on 300 genes per prompt
• Inference Capabilities: Capable of extrapolating beyond the 300 training genes to generate expression profiles for 1500-2000+ genes
• Max Generation: Up to 16,384 tokens (covering ~2000 genes).
• Model Trained by scBaseCount (https://github.com/ArcInstitute/arc-virtual-cell-atlas/tree/main/scBaseCount)

Usage

You can use this model with the transformers library or via the provided generation scripts.

Python API

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True
)

# Example Prompt (format depends on training data)
prompt = "Tissue: lung, Organ: lung, Disease: lung adenocarcinoma, Perturbation: unsure\nGenes:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# Generate
outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use

• Generating synthetic scRNA-seq or spatial transcriptomics data.
• Data augmentation for single-cell analysis.
• Imputation or study of gene expression patterns under specific biological conditions (tissue, disease, perturbation).

Files in this Repository

• model.safetensors: The model weights.
• config.json: Model configuration.
• tokenizer.json, vocab.json, merges.txt: Tokenizer files.
• generation_config.json: Default generation parameters.