Track2Map

Demo

Online reconstruction visualization

If the embedded video does not play in your browser, download the demo video.

Track2Map is an online stereo 3D Gaussian Splatting pipeline for deformable surgical scenes. It jointly refines camera trajectory and deformable anatomy, using dense 2D tracks to initialize deformation and to gate pose updates during camera-static periods.

Pose priors optional

Runs with clean, noisy, or absent camera pose priors.

Motion-aware SLAM

Reduces drift by avoiding pose updates when tissue/tool motion dominates.

Dense reconstruction

Produces metric deformable 3D Gaussian reconstructions from surgical stereo video.

Method

Tracking cues become geometry and pose constraints

Track

Dense 2D correspondences are estimated online between consecutive surgical frames.

Lift & gate

Tracks are lifted with stereo depth into 3D anchors, while flow-direction spread decides whether pose updates are safe.

Map

Anchors initialize deformation, and online 3DGS optimization refines the scene and the camera pose when enabled.

How Track2Map separates camera motion from tissue motion

We use dense 2D track directions as a lightweight surgical motion prior. Camera motion tends to produce a coherent direction distribution, while local tissue/tool motion produces a broad or multi-modal distribution. Pose optimization is enabled only when the circular standard deviation of track directions is below a threshold.

Flow direction distribution used to gate pose optimization — Flow-direction dispersion provides a compact signal for deciding when camera pose should be optimized.

Intuition

When the endoscope moves, tracked points tend to flow coherently. When the camera is static, tool interaction and tissue deformation produce mixed, local directions.

Gating rule

Track2Map checks whether most tracks move in a similar direction. If the direction spread is low, it updates the camera pose; if the spread is high, it freezes the pose and lets deformation explain the motion.

Why it matters

Freezing pose during high-dispersion periods prevents local tissue/tool motion from being absorbed into the camera trajectory, reducing drift during online mapping.

Results

Robust reconstruction and camera tracking on StereoMIS

27.58 PSNR, no pose prior

27.56 PSNR, heavy pose noise

0.0285 m ATE on StereoMIS

0.290° Relative rotation error

Reconstruction quality table on StereoMIS — Reconstruction quality on StereoMIS across absent, noisy, and clean pose-prior regimes.

Qualitative reconstruction under heavy pose noise — Under heavy pose noise, Track2Map preserves coherent anatomy while the pose-prior baseline accumulates visible geometric distortions.

Camera pose estimation table on StereoMIS — Camera pose estimation accuracy on StereoMIS. Lower is better for ATE and RPE.

Pose estimation comparison on StereoMIS — Track2Map follows the ground-truth trajectory more closely than Endo3R and EndoGSLAM-H across three StereoMIS sequences.

Scope and limitations

“Online” means frames are processed sequentially without future observations, not real-time latency.
The motion gate assumes camera-dominant and tissue/tool-dominant phases are separable; optical-axis motion remains a limitation.

Citation

BibTeX will be updated once Springer/arXiv metadata is available.