Spectral GS-SLAM

Observability-Aware, Degeneracy-Robust Tracking for Real-Time 3D Gaussian Splatting SLAM.

Edward Beng Wai Tan¹ Siew-Kei Lam¹ Dongshuo Zhang¹

¹Nanyang Technological University

IROS 2026

arXiv Code Demo BibTeX

our tracker on TUM nostructure / near nostructure perspective · real-time · RGB-D

Overview

Recent 3DGS-SLAM systems achieve real-time operation by replacing dense photometric optimisation with feature-matching or ICP-based tracking. These lightweight trackers can become unreliable in scenes with limited visual texture (where keypoint matching is sparse) or limited geometric structure (where the ICP Hessian becomes ill-conditioned).

We propose Spectral GS-SLAM, a tracking framework that integrates ICP with complementary feature-based constraints. A spectral analysis of the ICP Hessian identifies under-constrained directions, and a Gaussian-aware planarity score, modelled by the intrinsic covariance of 3D Gaussians, which gauges the macroscopic geometry of the visible scene. When degeneracy is detected, a lightweight visual-feature prior is fused only into the under-constrained sub-space, leaving the shared Gaussian map used for rendering untouched. Full derivation in the paper.

pipeline degeneracy → fusion → pose

Input RGB-D frame Source points μ^s, ORB features

→

Module 1 Degeneracy detection Spectral analysis of the ICP Hessian + Gaussian-normal planarity score

→

Module 2 Gaussian-aware fusion Anisotropic injection of feature prior into under-constrained eigen-directions

Demo

Live reconstructions on TUM RGB-D.

Each panel shows: RGB + tracked features and depth on top, the rendered map and the trajectory underneath, with the live planarity score overlaid. All sequences are tracked in real time.

notexture / near

low-texture

Rich geometry but limited visual texture. ICP carries the tracking and the prior is largely gated off.

nostructure / near

low-structure

A largely planar surface with a dominant surface normal. The planarity score stays high and the feature-based prior is fused into the under-constrained eigen-directions of the ICP Hessian.

fr1 / desk

standard

A standard TUM scene with mixed geometry and texture, with threshold crossing of the planarity gate.

Results

Tracking on TUM RGB-D.

ATE RMSE on the deliberately degenerate no-structure and no-texture splits, with whole-system FPS. Full results on Replica, standard TUM sequences, mapping fidelity, and ablations are in the paper.

TUM RGB-D · no-texture / no-structureATE RMSE (cm) ↓ · FPS ↑

Method	Tracker	% tracked	No Texture				No Structure				FPS
Method	Tracker	% tracked	near	n.val	far	f.val	near	n.val	far	f.val	FPS
SplaTAM non-RT	Photometric	100	2.27	2.81	4.54	4.14	4.49	3.65	9.49	5.32	0.54
MonoGS non-RT	Photometric	100	26.91	28.74	10.39	18.63	11.86	5.37	4.10	3.68	1.89
Photo-SLAM RT	ORB	68.4	×	×	×	×	2.27	3.31	5.36	3.83	31.48
GS-ICP SLAM RT	ICP	41.0	1.55	1.10	6.07	4.53	194.93	197.65	116.16	143.16	67.90
Ours RT	ICP + ORB	100	1.55	1.09	2.19	3.89	7.79	9.66	16.85	12.96	40.14

Best Second-best (underlined) Did not maintain tracking % tracked = fraction of poses within 1.0 m of ground truth

Mapping fidelity · no-texture / no-structurePSNR ↑ · SSIM ↑ · LPIPS ↓

Method	Metric	No Texture		No Structure
Method	Metric	near	far	near	far
MonoGS non-RT	PSNR	20.82	21.95	20.97	26.59
	SSIM	0.895	0.902	0.811	0.863
	LPIPS	0.447	0.363	0.337	0.220
SplaTAM non-RT	PSNR	26.71	30.70	20.71	21.36
	SSIM	0.924	0.949	0.827	0.903
	LPIPS	0.184	0.136	0.249	0.124
GS-ICP SLAM RT	PSNR	27.97	28.15	18.02	18.17
	SSIM	0.931	0.929	0.741	0.750
	LPIPS	0.135	0.137	0.507	0.452
Ours RT	PSNR	29.19	30.32	22.61	24.62
	SSIM	0.948	0.942	0.768	0.867
	LPIPS	0.138	0.127	0.260	0.163

BibTeX

Cite this work.

@misc{tan2026spectralgsslamobservabilityawaredegeneracyrobust,
  title         = {Spectral GS-SLAM: Observability-Aware, Degeneracy-Robust
                   Tracking for Real-Time 3D Gaussian Splatting SLAM},
  author        = {Edward Beng Wai Tan and Siew-Kei Lam and Dongshuo Zhang},
  year          = {2026},
  eprint        = {2606.21258},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2606.21258}
}

Acknowledgements

We thank the authors of GS-ICP SLAM, Photo-SLAM, SplaTAM, MonoGS, Point-SLAM and GS-SLAM for releasing their code, and the TUM RGB-D and Replica teams for the datasets used in this work.