DLSA

DLSA

Official implementation of DLSA: Dual-Level Semantic Alignment with Adaptive Multi-Scale Modeling for Referring Remote Sensing Image Segmentation, accepted to EMNLP 2026.

News

• August 2026: DLSA was accepted to EMNLP 2026.
• Source code is available in the GitHub repository, and trained models are available in this repository.

Models

We provide final checkpoints trained on RefSegRS and RRSIS-D through our Hugging Face model repository.

Place downloaded checkpoints under checkpoints/ as dlsa_refsegrs.pth and dlsa_rrsisd.pth. The public checkpoints contain the model parameters and configuration required for evaluation; optimizer states are intentionally omitted.

Framework

<p align="center"> <a href="assets/framework.pdf"> <img src="assets/framework.png" alt="DLSA framework" width="100%"> </a> </p>

DLSA performs semantic alignment at both the multi-scale visual encoding and mask decoding stages. The implementation is organized around the SgAMC modules in sgamc/ and the adaptive language-guided decoder modules in alti/.

Installation

The reference environment uses Python 3.7, PyTorch 1.13.1, torchvision 0.14.1, and CUDA 11.7.

conda env create -f environment.yml
conda activate dlsa

Alternatively, create a Python 3.7 environment, install PyTorch 1.13.1 with CUDA 11.7, and then run pip install -r requirements.txt.

Preparation

Datasets

Download RefSegRS and RRSIS-D from their official project pages. This repository does not redistribute either dataset. Arrange them as follows:

datasets/
|-- RefSegRS/
|   |-- images/
|   |-- masks/
|   |-- output_phrase_train.txt
|   |-- output_phrase_val.txt
|   `-- output_phrase_test.txt
`-- RRSIS-D/
    |-- rrsisd/
    |   |-- refs(unc).p
    |   `-- instances.json
    `-- images/
        `-- rrsisd/
            `-- JPEGImages/

The data roots can be changed through the DATA_ROOT environment variable in every provided script.

Initial Weights

Download the BERT-base-uncased files from the official BERT project to bert-base-uncased/. Download the Swin Transformer base checkpoint from the official Swin Transformer project and place it at:

pretrained_weights/swin_base_patch4_window12_384_22k.pth

The supplied DLSA checkpoints are sufficient for evaluation.

Training and Inference

Clone the GitHub repository, then train DLSA with:

bash scripts/run_refsegrs_pipeline.sh
bash scripts/run_rrsisd_pipeline.sh

Evaluate the released checkpoints:

bash scripts/test_refsegrs.sh
bash scripts/test_rrsisd.sh

Use SPLIT=val to evaluate the validation split. GPU IDs and paths can be overridden without editing the scripts, for example:

GPU=1 SPLIT=val CHECKPOINT=/path/to/model.pth \
  DATA_ROOT=/path/to/RefSegRS bash scripts/test_refsegrs.sh

Acknowledgements

This code is built on RMSIN. We thank the authors for making their work publicly available.

Citation

The final BibTeX entry will be added after the EMNLP 2026 proceedings are published.

License

This repository is released under the GNU General Public License v3.0. See LICENSE and THIRD_PARTY.md for details.