Long-term relocalization with a probabilistic topological map, for RGB-D, stereo and monocular cameras.

CROSS builds a map from a camera stream, saves it, and relocalizes later sessions in it, across changes in lighting, weather, season and furniture. The map is a graph of keyframes linked by odometry and by verified loop closures. CROSS tracks the camera pose as a Gaussian mixture over SE(3), so competing hypotheses stay alive until the evidence separates them. Relative poses come from a learned multi-view model (VGGT-Omega, fine-tuned for CROSS) or from feature matching with PnP on depth.

New here? Start with the getting-started guide. Every option is in the reference.

• Fourteen shipped modes: RGB-D, stereo or one camera, combined with external odometry (wheel, INS, a phone's VIO), a Basalt VIO file, CROSS's own IMU fusion, or DPVO visual odometry (no odometry input needed).
• Multi-hypothesis tracking: a Gaussian mixture over SE(3) with evidence-driven birth, realization and removal of hypotheses; a new session joins a map only when its placement is consistently supported.
• Verified loop closure: prior, in-pass and posterior chi-square tests at one confidence level, and pose-graph optimization (GTSAM) that runs in a background process on large maps.
• Change-robust relative poses: one forward pass of VGGT-Omega registers the current view against all retrieved keyframes; the metric scale comes from the stereo baseline, the depth sensor, the odometry or the map.
• Large maps: compact on-disk storage with lazy image loading, a retrieval index, windowed pose-graph optimization, a keyframe quality filter, and optional GNSS / compass anchoring.
• Remote sessions: the GPU work on a server, the odometry on the robot (the torch-free cross-edge package).
• Benchmark: one protocol (mapping accuracy, multi-session localization, relocalization success) on KITTI, OpenLORIS-Scene, ROVER and SimChange, with ORB-SLAM3, RTAB-Map, MASt3R-SLAM and VGGT-SLAM 2.0 as baselines.
• Optional packages: Gaussian-splatting reconstruction of a saved map (cross_world) and navigation on the same backbone (cross_nav).

Requirements: Linux, an NVIDIA GPU with CUDA, uv (the installer fetches it if missing). The installer creates .venv (Python 3.11), installs PyTorch, CROSS and GTSAM, and downloads the models.

git clone https://github.com/jiaming-ai/CROSS.git
cd CROSS
bash install.sh --stereo          # RGB-D modes + the VGGT-Omega modes (the usual choice)
source .venv/bin/activate

bash install.sh --help lists every option. Check the installation with python -m pytest -q (a few minutes).

# build a map of one session and save it
python run.py data/posed/office1-1 --mode rgbd_wheel --save-map maps/office/map.pkl
# relocalize a later session in it; write its poses in the map frame
python run.py data/posed/office1-2 --mode rgbd_wheel --load-map maps/office/map.pkl --save-poses office1-2.txt

run.py reads posed RGB-D folders, stereo folders, KITTI raw, TartanAir V2, Virtual KITTI 2, TUM RGB-D, OpenLORIS and Record3D files. The guide walks through this example on public data, explains the folder layout for your own recordings, and shows the Python API.

Choose the mode that matches your sensors with --mode <name> (configs/modes/<name>.yaml):

Stereo and single-camera modes run VGGT-Omega; the rgbd_ modes without ff match XFeat features with LightGlue and solve PnP on the depth image; mono_dpvo uses learned metric depth (Depth Anything 3). See USAGE.md for what each mode needs.

cross/             the library
  core/            system pipeline, hypotheses, loop closure, pose graph, planning, configuration
  cv/              relative pose estimation (PnP; VGGT-Omega with metric scale), features, detection
  db/              keyframe database, retrieval index, map storage
  mono/            monocular frontends (DPVO, Depth Anything 3), VGGT-Omega + IMU odometry
  imu/  geo/       IMU preintegration and scale; GNSS / compass anchoring
  remote/  rig/    split sessions (server / edge); two-phone stereo rig self-calibration
  dataloader/      dataset readers (posed and stereo folders, KITTI, TUM, OpenLORIS, R3D, ...)
  pipeline.py      modes: sensor setup x odometry source, configuration files
configs/           modes/ (one file per mode), base/ (shared pieces), overlays/ (optional layers)
run.py             run one sequence: map, relocalize, save
scripts/           map-and-relocalize harnesses, dataset converters, baselines, rig calibration, server
benchmark/         evaluation protocol, dataset preparation, job runner, results and page
edge/              cross-edge (robot side of remote sessions)
vggt_ft/           fine-tuning of VGGT-Omega (the recipe of our checkpoint)
cross_world/  cross_nav/   optional: 3D reconstruction, navigation
examples/          interactive demo, multi-session, save / load / plan
tests/             unit and integration tests (python -m pytest)
third_party/       vendored VGGT-Omega and Depth Anything 3

CROSS's code is released under the MIT license. Third-party components keep their own licenses:

• VGGT-Omega (third_party/vggt-omega, the released checkpoint, and our fine-tuned checkpoint derived from it): FAIR Noncommercial Research License. The stereo, rgbd_ff_* and single-camera modes use it.
• Depth Anything 3: code Apache-2.0 (third_party/depth-anything-3); the DA3-LARGE-1.1 weights that install.sh --mono downloads are CC BY-NC 4.0.
• DPVO (built by install.sh --mono): MIT (patches/DPVO-LICENSE).

The RGB-D modes with PnP use none of the non-commercial models.

If you use CROSS in your research, please cite:

@inproceedings{wang2026cross,
  title     = {Change-Robust Online Topological Memory for Long-Term
               Relocalization and Semantic Navigation},
  author    = {Wang, Jiaming and Chen, Jizhuo and Liu, Diwen and
               Ghotavadekar, Atharva and Da, Jiaxuan and K{\"a}stner, Linh
               and Soh, Harold},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  eprint    = {2605.02227},
  archivePrefix = {arXiv}
}