satvision-pix4d-cloud-base

SatVision-Pix4DCloud Base - Pre-Release

Pre-release: SatVision-Pix4D Base is an early research release intended for evaluation and experimentation. Model weights, preprocessing procedures, interfaces, and documentation may change in future releases.

SatVision-Pix4DCloud Base is a spatiotemporal foundation model for geostationary satellite observations, pretrained on sequences of all 16 GOES ABI spectral bands across 7 timesteps sampled every 20 minutes.

The model is part of the SatVision-Pix4DCloud project and is designed as a reusable encoder for downstream atmospheric and Earth-observation applications, including cloud characterization, convection analysis, three-dimensional cloud retrieval, and short-term weather nowcasting.

Unlike many Earth-observation representation models that primarily emphasize clear-sky surface information, SatVision-Pix4DCloud explicitly retains clouds and atmospheric variability as information-bearing features. This makes the model particularly suited to applications where cloud structure, evolution, and convection are central to the prediction problem.

Model Description

SatVision-Pix4DCloud uses a spatiotemporal Masked Autoencoder (MAE) training objective based on a Vision Transformer architecture.

The model learns representations from sequences of multispectral GOES Advanced Baseline Imager (ABI) Level-1B observations, allowing it to capture not only spatial and spectral structure, but also the temporal evolution of clouds and atmospheric systems.

Each training sample contains:

• Sensor: GOES Advanced Baseline Imager (ABI)
• Spectral channels: 16 ABI bands
• Temporal sequence: 7 timesteps
• Temporal spacing: 20 minutes
• Temporal coverage: 120 minutes from first to last observation
• Nominal tile size: 512 × 512 pixels
• Patch size: 16 × 16 pixels
• Mask ratio: 60%
• Encoder embedding dimension: 1024
• Transformer depth: 24 blocks
• Attention heads: 16
• MLP ratio: 4
• Decoder embedding dimension: 512
• Decoder depth: 8 blocks
• Decoder attention heads: 16

By learning jointly from the spectral, spatial, and temporal dimensions of ABI observations, SatVision-Pix4DCloud is designed to encode the evolution of atmospheric phenomena rather than treating satellite images as independent snapshots.

Why SatVision-Pix4DCloud?

Geostationary satellites provide a perspective fundamentally different from conventional polar-orbiting Earth-observation datasets.

GOES ABI repeatedly observes the same region at high temporal frequency, providing direct information about the evolution of clouds, convection, atmospheric moisture, and other rapidly changing Earth-system processes.

SatVision-Pix4DCloud takes advantage of this capability by ingesting seven consecutive ABI observations sampled every 20 minutes. Rather than learning exclusively from individual satellite images, the model learns from approximately two hours of atmospheric evolution.

A central motivation of SatVision-Pix4DCloud is therefore to move beyond representations optimized primarily for static or clear-sky surface properties and toward representations capable of describing the dynamic atmosphere.

Input Data

SatVision-Pix4DCloud expects spatiotemporal multispectral ABI observations prepared consistently with the SatVision-Pix4DCloud preprocessing pipeline.

Conceptually, an input sample can be represented as:

[B, T, C, H, W]

where:

T = 7 timesteps
C = 16 ABI spectral bands
H = 512
W = 512
Δt = 20 minutes

Therefore, each sample contains:

7 timesteps × 16 ABI bands × 512 × 512 pixels

with the temporal sequence:

t₀
t₀ + 20 min
t₀ + 40 min
t₀ + 60 min
t₀ + 80 min
t₀ + 100 min
t₀ + 120 min

This temporal context enables the encoder to learn representations associated with atmospheric evolution, including cloud growth, organization, displacement, and convective development.

Users should reproduce the preprocessing and normalization conventions used during pre-training before applying the released weights to new imagery.

Architecture

The model processes a multispectral temporal sequence rather than an isolated image:

       GOES ABI temporal sequence
  7 timesteps × 16 spectral bands
        sampled every 20 minutes
                  │
                  ▼
         512 × 512 spatial tiles
                  │
                  ▼
           Patch embedding
                  │
                  ▼
         Random patch masking
             (60% masked)
                  │
                  ▼
         Vision Transformer
              Encoder
                  │
                  ▼
       Spatiotemporal atmospheric
             representation
                  │
            ┌─────┴─────┐
            │           │
            ▼           ▼
       MAE Decoder   Downstream
      (pre-training)    Heads

The pretrained representation encodes information across three complementary dimensions:

Spectral — relationships among the 16 ABI channels.

Spatial — cloud morphology, texture, gradients, and Earth-system structures within each 512 × 512 tile.

Temporal — changes occurring across seven observations spanning two hours.

This combination is particularly important for atmospheric applications, where the evolution of a feature can be as informative as its instantaneous appearance.

Pre-training Data

SatVision-Pix4DCloud is trained using multispectral, multitemporal GOES ABI Level-1B imagery generated through the SatVision-Pix4DCloud data pipeline.

The pipeline constructs temporal windows from ABI observations and selects seven-timestep sequences sampled at 20-minute intervals for model pre-training.

Each resulting training sample therefore combines:

16 ABI spectral channels
        ×
7 temporal observations
        ×
512 × 512 spatial pixels

The pipeline additionally supports targeted sampling strategies such as convection-stratified sampling, allowing the training corpus to better represent meteorologically important atmospheric states and cloud systems.

Intended Uses

The explicit temporal dimension makes SatVision-Pix4DCloud particularly relevant for downstream applications involving atmospheric evolution, including:

• convective initiation and development,
• short-term weather nowcasting,
• cloud motion and evolution,
• convective system identification,
• cloud segmentation,
• cloud-property retrieval,
• three-dimensional cloud reconstruction,
• atmospheric state retrieval,
• severe-weather analysis,
• spatiotemporal feature extraction,
• multimodal Earth-system learning.

For applications such as nowcasting, the seven-frame input provides the downstream model with information about how the observed atmosphere has evolved during the preceding two hours, rather than only its state at a single observation time.

Loading the Model

The model checkpoint can be downloaded using the Hugging Face Hub:

from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="nasa-cisto-data-science-group/satvision-pix4d-base",
    filename="<checkpoint-filename>",
)

print(checkpoint_path)

Note: SatVision-Pix4DCloud is currently implemented using the architecture and training utilities provided by the SatVision-Pix4DCloud repository. It is not currently packaged as a native transformers.AutoModel architecture.

Clone the implementation repository with:

git clone https://github.com/nasa-nccs-hpda/satvision-pix4d-cloud.git
cd satvision-pix4d-cloud

Install the project dependencies using the environment appropriate for your system.

Example Model Configuration

The base architecture is configured approximately as follows:

MODEL:
  TYPE: satmae

  MAE_VIT:
    PATCH_SIZE: 16
    IN_CHANS: 16
    EMBED_DIM: 1024
    DEPTHS: 24
    NUM_HEADS: 16
    MLP_RATIO: 4.0

    DECODER_EMBED_DIM: 512
    DECODER_DEPTH: 8
    DECODER_NUM_HEADS: 16

DATA:
  IMG_SIZE: 512
  MASK_RATIO: 0.6

Refer to the GitHub repository for the current implementation and configuration files.

Input Data

The model expects multispectral GOES ABI observations prepared consistently with the SatVision-Pix4DCloud preprocessing pipeline. The architecture supports both single-timestep and multi-timestep inputs, allowing it to be used for spatial-spectral or spatiotemporal applications.

For the released SatVision-Pix4DCloud Base model, pre-training uses sequences of 7 timesteps sampled every 20 minutes, with all 16 ABI spectral bands at each timestep. Each spatial tile has a size of 512 × 512 pixels.

Conceptually, an input batch can be represented as:

[B, T, C, H, W]

where:

T = number of timesteps
C = 16      # ABI spectral bands
H = 512     # tile height
W = 512     # tile width

The temporal dimension T is flexible. For example:

T = 1   # single-timestep input
T = 7   # pre-training configuration

For the 7-timestep pre-training configuration, observations are sampled at 20-minute intervals:

t₀
t₀ + 20 min
t₀ + 40 min
t₀ + 60 min
t₀ + 80 min
t₀ + 100 min
t₀ + 120 min

This configuration spans 2 hours from the first to the final observation, enabling the model to learn spectral, spatial, and temporal representations of atmospheric structure and evolution while retaining the flexibility to operate on individual ABI observations.

Users should reproduce the preprocessing and normalization conventions used during pre-training before applying the released weights to new imagery.

Self-Supervised Objective

SatVision-Pix4DCloud uses masked image modeling.

Given an ABI observation (x), the image is divided into spatial patches. A subset of patches is removed from the encoder input:

[ x_{\mathrm{visible}} = M(x) ]

where (M) represents the random masking operation.

The encoder produces latent representations from the visible patches:

[ z = E(x_{\mathrm{visible}}) ]

and the MAE decoder reconstructs the masked image content:

[ \hat{x} = D(z) ]

Training minimizes reconstruction error over masked regions.

Because no semantic labels are required, large archives of satellite observations can be used directly for representation learning.

Downstream Transfer

The principal product of SatVision-Pix4DCloud pre-training is the encoder representation.

A typical downstream workflow is:

Pretrained SatVision-Pix4DCloud encoder
                │
                ▼
         Feature representation
                │
       ┌────────┼─────────┐
       ▼        ▼         ▼
 Segmentation Retrieval Nowcasting
    head       head       head

Depending on the application, researchers may:

1. freeze the encoder and train only the downstream head,
2. fine-tune the entire network,
3. fine-tune selected transformer blocks,
4. use SatVision-Pix4DCloud features as inputs to another model.

Training Infrastructure

The SatVision-Pix4DCloud software is designed for large-scale GPU and HPC execution and has been developed on NASA computing infrastructure.

The training implementation supports capabilities including:

• distributed GPU training,
• DeepSpeed,
• mixed BF16 precision,
• gradient checkpointing,
• large-scale ABI data loading,
• containerized execution.

These capabilities are implementation details rather than requirements for using the pretrained encoder.

Limitations

SatVision-Pix4DCloud representations reflect the characteristics and sampling distribution of the satellite observations used during pre-training.

Users should consider several limitations:

• The model is designed around GOES ABI imagery and should not be assumed to transfer directly to unrelated sensors without evaluation.
• ABI spectral channels have different spatial resolutions before preprocessing and resampling.
• Representation quality may vary geographically, seasonally, and across atmospheric regimes.
• Convective sampling can intentionally alter the distribution of training examples relative to the natural frequency of atmospheric states.
• The base model does not itself produce meteorological forecasts or physical atmospheric variables.
• Downstream models require independent validation for the intended scientific application.
• Predictions derived from this model should not be treated as operational weather guidance without task-specific validation.

Research Status

SatVision-Pix4DCloud is an active research project.

The software ecosystem includes development of capabilities for:

• large-scale ABI dataset construction,
• temporal satellite representation learning,
• convection-aware sampling,
• cloud-focused learning,
• downstream atmospheric retrieval,
• three-dimensional cloud reconstruction,
• and weather nowcasting.

Interfaces, checkpoints, preprocessing procedures, and model configurations may evolve as development continues.

Repository

Source code, training infrastructure, and data-generation workflows are available at:

SatVision-Pix4DCloud https://github.com/nasa-nccs-hpda/satvision-pix4d-cloud

Model

Hugging Face model repository:

nasa-cisto-data-science-group/satvision-pix4d-base https://huggingface.co/nasa-cisto-data-science-group/satvision-pix4d-cloud-base

License

The SatVision-Pix4DCloud software is released under the Apache License 2.0.

Please verify the license information associated with individual datasets and downstream products independently.

Citation

A formal citation for SatVision-Pix4DCloud will be added as associated publications become available.

If you use the model before a formal citation is released, please cite the software repository and model checkpoint:

@software{satvision_pix4d,
  title  = {SatVision-Pix4DCloud},
  author = {{NASA CISTO Data Science Group}},
  url    = {https://github.com/nasa-nccs-hpda/satvision-pix4d-cloud},
  year   = {2026}
}

Acknowledgments

SatVision-Pix4DCloud is developed within NASA's scientific computing and data-science ecosystem with the goal of enabling scalable artificial intelligence research using geostationary Earth observations.

Development leverages NASA high-performance computing resources and large-scale archives of GOES ABI observations.

Disclaimer

This model is a research product and is not an operational weather forecasting system.

Users are responsible for validating model performance, scientific appropriateness, and uncertainty for their specific application.