TerraFM

TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation

<p align="center"> <img src="https://i.imgur.com/waxVImv.png" alt="Oryx TerraFM"> </p>

---

📢 Latest Updates

• Jun-09-25: 🚀 Initial release of TerraFM codebase and pretrained models
• Jun-09-25: 📄 Paper released on arXiv: arxiv link. 🔥🔥

---

🌍 Overview

TerraFM is a scalable foundation model designed for unified processing of multisensor Earth Observation (EO) data. Built on a ViT backbone and trained over 18.7M tiles (~23T pixels) from Sentinel-1 SAR and Sentinel-2 optical imagery, TerraFM unifies modality-specific inputs using:

• 🧩 Modality-specific patch embeddings
• 🌀 Adaptive cross-attention fusion
• 🎯 Dual-centering regularization for long-tailed distributions

TerraFM sets a new benchmark on GEO-Bench and Copernicus-Bench, demonstrating strong generalization across geographies, modalities, and tasks — including classification, segmentation, and landslide detection.

---

🔬 Key Features

<p align="center"> <img src="images/spider_gb.jpg" alt="TerraFM Architecture" width="500"/> </p>

• Multimodal Pretraining: Uses Sentinel-1 (SAR) and Sentinel-2 (L1C, L2A) as natural augmentations.
• Large-Scale Dataset: Trained on 18.7M global tiles from the Major-TOM dataset.
• Cross-Attention Fusion: Dynamically aggregates information across sensors at patch level.
• Dual-Centering: Mitigates long-tailed land cover bias using ESA WorldCover statistics.
• Benchmark SOTA: Outperforms prior FMs (Galileo, Prithvi, DOFA) across multiple EO tasks.

---

🧱 Architecture

<p align="center"> <img src="images/arch.jpg" alt="TerraFM Architecture" width="700"/> </p>

Overall architecture of TerraFM. It unifies student-teacher contrastive framework with modality augmentation with cross-attention fusion, and a new dual centering regularization. TerraFM is founded on ViT backbone and is trained on 18.7M globally distributed samples for pre-training and utilizes large-tile inputs for encoding broader spatial context. For illustration, RGB channels from S2-L2A and S2-L1C are selected, and S1 is visualized using a false-color RGB composite.

---

🧠 Model Zoo

---

🛠 Usage

TerraFM can be used directly via the terrafm.py module, which provides standalone implementations of the TerraFM-Base and TerraFM-Large models for easy integration into any codebase.

from terrafm import terrafm_base, terrafm_large
import torch

# Simulated input: 1 sample, 12 channels, 224×224 resolution (e.g., Sentinel-2 L2A)
x = torch.randn(1, 12, 224, 224)

# Load TerraFM-Base model
model = terrafm_base()

# Load pretrained weights (e.g., TerraFM-B.pth)
state_dict = torch.load("TerraFM-B.pth", map_location="cpu")
msg = model.load_state_dict(state_dict, strict=False)

# Forward pass
y = model(x)
print(f"Output shape: {y.shape}")

---

📊 Results

🔍 k-NN Classification Results

We evaluate image classification using k-nearest neighbors (kNN) and report Top-1 accuracy for all single-label tasks. For the multilabel BigEarthNet benchmark, we report the F1 score.

🛰 Copernicus-Bench

Comparison of TerraFM with existing supervised and self-supervised methods on Copernicus-Bench.
Metrics include OA (Overall Accuracy), mAP (mean Average Precision), and mIoU (mean Intersection over Union).

🧪 GEO-Bench Performance

Performance comparison on GEO-Bench for both classification (Top-1 Accuracy), segmentation (mIoU), and F1 score (for m-BigEarthNet).
TerraFM achieves state-of-the-art results across multiple datasets, outperforming previous foundation models.

🌋 Landslide Detection (Landslide4Sense)

Landslide detection performance on the Landslide4Sense test set.
Despite having significantly fewer parameters (120M vs. 300M), TerraFM achieves higher overall segmentation performance, especially for landslide regions.

<p align="center"> <img src="images/ls4s_qual.jpg" alt="Landslide Detection" width="700"/> </p> ---

📜 Citation

If you find our work and this repository useful, please consider giving our repo a star and citing our paper as follows:

@article{danish2025terrafmscalablefoundationmodel,
      title={TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation}, 
      author={Muhammad Sohail Danish and Muhammad Akhtar Munir and Syed Roshaan Ali Shah and Muhammad Haris Khan and Rao Muhammad Anwer and Jorma Laaksonen and Fahad Shahbaz Khan and Salman Khan},
      year={2025},
      eprint={2506.06281},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.06281}, 
}

📨 Contact

If you have any questions, please create an issue on this repository or contact at muhammad.sohail@mbzuai.ac.ae.