pasta-tumor

PASTA Tumor Microenvironment Pathway Prediction Models

This repository contains PASTA (PAthological Spatial Transcriptome-guided pAradigm) models trained for predicting tumor-related pathway scores from H&E-stained pathology images.

Model Description

PASTA is a deep learning framework that can predict neuron-related pathway spatial patterns directly from histology images. These benchmark models were trained on HEST-library human data with different foundation model backbones.

Model Architecture

• Base Architecture: Modified Dense Prediction Transformer (DPT)
• Task: Tumor-related pathway score prediction
• Input: H&E-stained neuropathology image patches (224×224 pixels); Paired enriched pathway scores from ST data(optional)
• Output: Super-resolution spatial pathway maps

Available Models

This repository includes 3 models:

• UNI_14_pos.pt - UNI backbone, Gene set: Tumor_default_14
• Phikonv2_14_no_pos.pt - Phikonv2 backbone, Gene set: Tumor_default_14
• UNI_16_no_pos.pt - UNI backbone, Gene set: Tumor_default_16

Notes on variants:

• Numbers (14, 16) indicate different gene sets
• pos: Trained with coarse-grained pathway score/gene expression as positional embeddings. In these variants, you can use coarse-grained pathway scores/gene expression as an optional input.
• no_pos: Trained without pathway scores.

Usage

🌟 To use PASTA models for inference, please follow the code in PASTA codebase and confirm that you have the access to related backbone foundation models.

Loading and Using the Model

import torch
from pasta.model import PASTA
from huggingface_hub import hf_hub_download
from PIL import Image
import torchvision.transforms as transforms

# Download model checkpoint
model_path = hf_hub_download(
    repo_id="mengflz/pasta-tumor",
    filename="UNI_14_pos.pt"
)

# Initialize PASTA model
model = PASTA(
    model_name='UNI',
    pathway_dim=14,
    freq_flag=True,
    N=128,
    scale=1.0
)

# Load trained weights
checkpoint = torch.load(model_path, map_location='cpu')
model.load_state_dict(checkpoint, strict=False)
model.eval()

# Prepare input image
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], 
                       std=[0.229, 0.224, 0.225])
])

image = Image.open("path/to/tumor_image.png")
input_tensor = transform(image).unsqueeze(0)

# Run inference
with torch.no_grad():
    feat, feat_mean, feat_recon = model(input_tensor)
    # feat: spatial feature map [B, 14, H, W]
    # feat_mean: class logits [B, 14]
    # feat_recon: reconstructed image [B, 3, H, W]

Training Details

Please refer to the original PASTA paper or PASTA Github repository for more details.

Citation

If you use these models in your research, please cite:

License

This project is licensed under GPL-3.0. The underlying foundation models (UNI, PhikonV2) have their own licenses.

Contact

For questions and issues, please open an issue on the repository.