attention-readout-monkey-v4

Monkey V4 Data Driven + Attention Readout Model Card

Neural Encoding model for Macaque V4. The model is a combination of a data driven core and an attention readout layer.

<p align="center"><img src="./assets/schematic.png" width="100%" alt="Data Driven V4 Schematic" /></p>

Model Details

Model Description

This model is a combination of a data driven core and an attention readout layer. The data driven core is a convolutional neural network and the attention readout layer is a multihead attention layer with each head trained to predict the firing rates of a neuron in Macaque V4.

Model Sources

For research purposes, we recommend our nnvision Github repository (https://github.com/sinzlab/nnvision), which contains the code for the model defintions and training.

• Repository: https://github.com/sinzlab/nnvision
• Paper: https://www.biorxiv.org/content/10.1101/2023.05.18.541176v1

Intended Use

The model is intended for research purposes only.

Model Use

The model can be used to predict the firing rates of neurons in Macaque V4 given an image.

nnvision

The model can be used in Python with the nnvision package.

import torch
from nnvision.models.trained_models.v4_data_driven import v4_multihead_attention_ensemble_model

input_image = torch.rand(1, 100, 100)
firing_rate = v4_multihead_attention_ensemble_model(input_image, data_key="all_sessions")

energy-guided diffusion

The model can be used in Python with the energy-guided-diffusion package.

from egg.models import models
model = models['data_driven']['train']