WAVE-models

WAVE-models

Description

WAVE models are generative models designed to decode natural images from fMRI data. We utilize contrastive learning and diffusion models, trained using the PyTorch library. This suite includes both universal models trained across all subjects and subject-specific models, particularly for the CSI1 subject within the BOLD5000 dataset. Pre-trained models and additional resources are also available. Check our paper here: https://arxiv.org/abs/2411.07121

Model Structure

Universal Model

• All: A universal model trained on data from all subjects.

Subject-Specific Model

• CSI1: A model specifically trained on the CSI1 subject.

Model Folder Structure

• contrastive/: Contains the contrastive learning models.
• decode/: Contains the diffusion models.

Model Files

• */contrastive/model_last_prompt.bin: A model integrating contrastive learning with prompt.
• */contrastive/model_last_vis-mask.bin: A model integrating contrastive learning with prompt and visual network masking.
• */decode/model_last_prompt_vd.bin: Versatile diffusion model fine-tuned on the contrastive prompt learning model.
• */decode/model_last_prompt_vis-mask_vd.bin: Versatile diffusion model fine-tuned on the contrastive prompt and visual network masking model.

Pre-trained fMRI Model

Available at:

• upstream/GPT/pytorch_model.bin: Pre-trained GPT model for fMRI data from this GitHub repository.

Additional Resources

DiFuMo Atlas

• labels_1024_dictionary.csv: Contains labels for the DiFuMo atlas.

Yeo 7 Atlas

• index_yeo7.json: Labels for DiFuMo atlas indices associated with visual networks.

For further details on the models and applications, refer to the GitHub repository.