merlin-mapanything-student

MERLIN — MapAnything student

A 230M distilled student of MapAnything that turns a single monocular RGB frame into metric depth + pose, running live at 16-17 FPS (TensorRT INT8) on a $249 Jetson Orin Nano 8GB. Code: github.com/ShayManor/Merlin.

Which checkpoint to use

student_nf.pt for navigation, student_v3.pt for depth fidelity, and note that the two are not interchangeable.

The navigation failure this model addresses is a data gap, not a modelling one. Training poses were sampled as navigable, so they stand clear of every surface and almost no training pixel sat inside a metre. Fine-tuning on deliberately rendered near-field frames cuts the 0.2-0.5 m signed depth bias from +0.193 m to +0.035 m, and paired closed-loop success on the open-space obstacle benchmark rises from 0.469 to 0.703 (+0.234, 95% CI [+0.047, +0.422], n=32 layouts). Evaluation scenes were held out of fine-tuning.

Two caveats. The 16-17 FPS, INT8 and fidelity figures above were all measured on student_hab_nav.pt; re-quantize and re-profile before any deployment claim leans on student_nf.pt. And the closed-loop result is same-simulator, trained and evaluated in Habitat on ReplicaCAD, so it is not yet evidence the gain survives a real camera.

Longer fine-tunes on the same data were tried and are worse, which is why they are not published here: past ~3000 steps whole-frame accuracy keeps improving while the voxel map degrades, and a 8000-step run blocked 59% of a collision-free route against ground truth's 10%. Select this checkpoint class by the map gate, not by the loss.