TerraMind-1.0-Tokenizer-S1RTC

TerraMind 1.0 S-1 RTC Tokenizer

TerraMind is the first multimodal any-to-any generative foundation model for Earth Observation jointly developed by IBM, ESA, and Forschungszentrum Jülich. The model is pre-trained using FSQ-VAE tokens as targets. This tokenizer encodes and decodes Sentinel-1 RTC satellite images for the TerraMind model.

The tokenizer uses FSQ with five dimensions and a codebook size of 15'360 tokens. The decoding process uses diffusion steps for the reconstruction. The model was pre-trained for up to 100 epochs on nine million S-1 RTC images from the TerraMesh dataset.

Usage

The tokenizer is fully integrated into the fine-tuning toolkit TerraTorch. You can initialize the pre-trained tokenizer with:

from terratorch.registry import FULL_MODEL_REGISTRY
model = FULL_MODEL_REGISTRY.build('terramind_v1_tokenizer_s1rtc', pretrained=True)

Once the model is build, it can be used to encode image and decode tokens. The number of diffusion steps is defined with timesteps. Increasing the diffusion steps adds more details to the reconstruction which can also lead to hallucinations.

# Encode image
_, _, tokens = model.encode(s1rtc_tensor)
# Decode tokens
reconstruction = model.decode_tokens(tokens, verbose=True, timesteps=10)
# Encode & decode
reconstruction = model(s1rtc_tensor)

This tokenizer is automatically loaded with TerraMind generation models like terramind_v1_base_generate, see here for details.

We provide example code for the tokenizer at https://github.com/IBM/terramind.

Feedback

If you have feedback or any questions, please start a discussion in this HF repository or submitting an issue to TerraMind on GitHub.

Citation

If you use TerraMind in your research, please cite our TerraMind paper.

@article{jakubik2025terramind,
  title={TerraMind: Large-Scale Generative Multimodality for Earth Observation},
  author={Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
  journal={IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2025}
}