project_nott

NÓTT — Thermal Image Refinement for Monocular ORB-SLAM3

Part of the ANIMA Perception Suite by Robot Flow Labs.

Models

v3_sol — VIVID++ pretrained + SOL synthetic fine-tuned (BEST)

Fine-tuned on robotflowlabs/anima-thermal-synthetic (26K COCO synthetic + 399 VIVID real pairs). 8-GPU DDP with frozen BatchNorm.

• ARE: 0.174 | val_loss: 0.079

v2 — VIVID++ pretrained (baseline)

Trained on VIVID++ dataset (71,917 thermal/depth paired frames, 24 sequences).

• ARE: 0.106 | val_loss: 0.034

Exported Formats

Architecture

T-RefNet — U-Net encoder-decoder with ConvGRU recurrent bottleneck (2M params, 8MB).

Paper

Thermal Image Refinement with Depth Estimation using Recurrent Networks for Monocular ORB-SLAM3 arXiv:2603.14998 — Şahin, Pham, Dang, Yegenoglu, Kayacan (2026)

Training Pipeline

VIVID++ (71K real thermal) → pretrain (lr=1e-3) → v2 (ARE=0.106)
    ↓
SOL synthetic (26K COCO + 399 real) → fine-tune (lr=1e-5, frozen BN, 8 GPU DDP) → v3_sol (ARE=0.174)

Usage

import torch
from anima_nott.thermal_refinement import ThermalRefinementNet

model = ThermalRefinementNet(in_channels=1, base_channels=32, num_levels=3, gru_layers=2)
state = torch.load("pytorch/nott_v3_sol.pth", weights_only=True)
model.load_state_dict(state)
model.eval()

thermal = torch.randn(1, 1, 256, 320)  # normalized [0,1]
refined, hidden = model(thermal)

Defense Application

Low-cost thermal SLAM for GPS-denied, low-light UAV navigation. Non-radiometric thermal cameras (~$150 FLIR Lepton 3.5). Target: <0.4m trajectory error, 25+ FPS on Jetson Xavier.

License

Apache-2.0 — Robot Flow Labs / AIFLOW LABS LIMITED