timesformer_GP_scroll1

Model Card for timesformer_GP_scroll1

The grandprize winning model of the Vesuvius Challenge of 2023.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

The grandprize winning model of the Vesuvius Challenge of 2023.
The model features a small TimeSformer architecture trained on image segmentation task to detect ink in 3d images.
This model takes as input the 3d image and outputs a 2d map of ink detections, roughly 1/16 the size of the input.

• Developed by: Youssef Nader as part of the Grandprize Winning Team
• Model type: TimeSformer
• License: MIT

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/younader/Vesuvius-Grandprize-Winner [archived]
  Active development resumed here: https://github.com/ScrollPrize/villa

How to Get Started with the Model

Make sure to have the dependencies installed, namely transformers and <a href="https://github.com/lucidrains/TimeSformer-pytorch">Timesformer package</a>

pip install -U transformers timesformer-pytorch

Next you can run the model as follows:

from transformers import AutoModel
model = AutoModel.from_pretrained("YoussefMoNader/timesformer_GP_scroll1", trust_remote_code=True)

the model expects a (B,1,26,64,64) tensor

<!-- ## Training Details <!-- ### Training Data <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Hardware

The model was trained on 4xH100 for 8 hours. This model was trained for 12 epochs on total, a single epoch takes around 45 mins using the old script train_timesformer_og.py

<!-- ## Technical Specifications [optional] ### Model Architecture and Objective [More Information Needed] ### Compute Infrastructure [More Information Needed] #### Hardware [More Information Needed] --> <!-- #### Software [More Information Needed] --> <!-- ## Citation [optional] <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> <!-- **BibTeX:** [More Information Needed] **APA:** [More Information Needed] ## Glossary [optional] <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> <!-- [More Information Needed] ## More Information [optional] [More Information Needed] ## Model Card Authors [optional] [More Information Needed] ## Model Card Contact [More Information Needed] -->