MONSTER Dataset

Robustness analysis and benchmarking for monocular SLAM—standard datasets plus controlled, isolated failure modes.

Explore the Dataset Paper (PDF) ArXiv GitHub

Teaser image placeholder

Put teaser.jpg in /static

Teaser idea: 3D viewer screenshot + birdseye overlay + metric table snippet.

SLAM algorithms evaluated

106

benchmark sequences

isolated challenge sources

difficulty levels per challenge

50x

runs per sequence

Abstract

Simultaneous Localization and Mapping (SLAM) is essential for autonomous robots, yet state-of-the-art visual SLAM algorithms still fail in many real-world environments. Existing benchmarks evaluate scenarios where multiple confounding factors occur simultaneously (e.g., motion blur, poor lighting, and dynamic objects), which makes it hard to identify actionable causes of performance degradation. This project performs a large-scale robustness analysis of 8 state-of-the-art monocular SLAM algorithms across sequences from public datasets and our MONSTER dataset (MONocular Synthetic TEsting of Robustness). MONSTER isolates individual sources of performance degradation and provides progressive difficulty levels, enabling precise identification of algorithm-specific weaknesses. We release the dataset, collected results, and an interactive visualizer to support reproducible research and accelerate progress toward robust real-world monocular SLAM deployment.

Highlights

Controlled robustness testing

Isolate a single failure mode at a time to pinpoint why algorithms break (instead of guessing from confounded scenes).

Interactive visual analytics

3D trajectories + synchronized video + metrics tables + birdseye projections—built for fast qualitative + quantitative inspection.

Reproducible evaluation protocol

Consistent runs/settings and repeat trials to capture stochasticity and summarize statistically meaningful trends.

Interactive Dataset Visualizer

A dashboard for exploring trajectories, videos, and performance metrics across datasets, variants, and algorithms.

Open Visualizer

Synchronized 3D + Video

Inspect estimated trajectories and scene geometry alongside the reference video in lockstep.

Comparative results table

Browse APE/RPE-style metrics grouped by algorithm family with filtering and highlighting.

Birdseye projection

Project trajectories onto a top-down reference image for a clear spatial overview (zoomable).

Custom uploads

Overlay your own trajectories for quick comparisons against published baselines.

Resources

Links for the paper, code, dataset downloads, and reproducibility.

Tip

If you want the home page to feel like an academic project site, add: 1 teaser image, 2 a one-paragraph "Method / Dataset design" section, and 3 a small "News" section with dates.

Cite

If you use MONSTER or the benchmark results in your research, please cite:

@article{monster2025,
  title   = {Robustness Analysis of Monocular SLAM Systems: Current Limitations and Future Research Directions},
  author  = {People's Names are Anonymized for Review},
  journal = {IEEE Transactions on Robotics},
  year    = {2025}
}