BAD SLAM performs real-time direct bundle adjustment over surfels and keyframes and introduces a well-calibrated, synchronised global-shutter RGB-D benchmark, showing direct RGB-D SLAM is highly sensitive to rolling shutter, RGB-depth synchronisation and calibration errors.

技術屬性

欄位內容為文獻擷取紀錄的原文用語(英文),以原文為據;「未查證」表示本研究尚未讀到該資訊,不代表該方法不具備此能力。

BAD SLAM 的技術屬性
感測輸入RGB-D
原文測試平台custom multi-camera rig with up to eight synchronized global-shutter cameras (two colour and two infrared cameras form the RGB-D sensor; four more used only for SfM ground truth) and an Asus Xtion Live Pro used solely as infrared pattern emitter, tracked by a Vicon system; carrying mode not stated in paper or supplement、public TUM RGB-D sequences (Kinect v1)、synthetic renders of TUM RGB-D reconstructions
狀態估計Alternating direct BA (Alg. 1): Gauss-Newton on a cost of Tukey-weighted point-to-plane residuals and Huber-weighted photometric gradient residuals (photometric weight 1e-2), each normalised by a stereo depth-noise model or an empirical sigma; surfels move only along their normals (joint 2x2 solve of offset and descriptor per surfel); independent per-keyframe SE(3) pose updates; optional intrinsics and per-pixel depth-deformation calibration solved with the Schur complement; interleaved discrete surfel creation, merging, outlier deletion and radius update. A PCG Gauss-Newton solver was slightly worse in the ablation.
資料關聯Surfel to pixel correspondences by projecting surfel centres into every keyframe, kept only if the pixel has depth, the Tukey weight of the geometric residual is positive and normals agree; photometric residual compares the surfel descriptor with the gradient magnitude sampled at the surfel centre and two disc boundary points. Front-end odometry: direct photometric and geometric SE(3) alignment of each frame to the last keyframe using intensity gradients.
時間表示discrete poses; every 10th frame becomes a keyframe; benchmark colour and depth are captured at the same instants, so no temporal interpolation between them is needed
去畸變不適用 (rolling shutter avoided by global-shutter, synchronised benchmark cameras rather than modelled)
迴圈閉合bag-of-words detection with binary features; relative pose from keypoint matches refined by direct alignment and checked for consistency against neighbouring keyframes m-1 and m+1; then pose-graph optimisation (Sec. 4; Fig. 2)
全域最佳化pose-graph optimisation on loop detection, plus direct BA back-end over keyframe poses, surfels and camera intrinsics (Fig. 2; Sec. 3-4)
地圖表示surfels (oriented discs with centre, normal, radius and a scalar gradient descriptor) created on 4x4 pixel cells of each keyframe, plus keyframes storing raw RGB-D; about 335,000 surfels for the Fig. 1 scene
先驗資訊none
可輸出幾何surfel map and keyframe trajectory
計算需求Intel Core i7 6700K with MSI GeForce GTX 1080 Gaming X 8G; BA on GPU with CUDA 8.0; with about 27 Hz input and one keyframe per 10 frames, 370 ms of BA time per keyframe are available and further BA iterations are skipped in real-time mode; odometry time negligible; memory grows linearly with keyframe count

使用設備

原文使用的感測器、運算硬體與載具(equipment)。型號保留原文寫法,連結到設備頁中同一型號的歸併名稱;角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測(reference or ground truth)與比較對象設備。

原文使用的設備
類別型號(原文寫法)角色資料集原文規格出處
慣性量測單元(IMU)IMU (model not stated in paper or supplement)資料集感測器ETH3D SLAM benchmark (this paper)benchmark stated to provide full IMU data(Schöps et al., 2019, Sec. 2, Table 1)
相機four additional synchronized cameras on the rig參考或真值量測ETH3D SLAM benchmark (this paper)used only to localise training sequences whose ground truth comes from Structure-from-Motion(Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1)
雙目相機two synchronized global-shutter colour cameras (front-facing stereo pair) on a custom multi-camera rig similar to Gohl et al. [22]資料集感測器ETH3D SLAM benchmark (this paper)one camera supplies the RGB image of the RGB-D frames; both usable for stereo SLAM; raw Bayer images debayered, flat-field corrected, no white balancing(Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1, 4.1, Fig. 2)
雙目相機two synchronized global-shutter infrared cameras (stereo pair below the colour cameras) on the same rig資料集感測器ETH3D SLAM benchmark (this paper)active stereo depth by PatchMatch stereo with ZNCC cost and 11x11 window, reprojected to the colour camera; colour and depth recorded at exactly the same time(Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1, 4.6)
雙目相機Intel D435比較對象設備未標示infrared emitter considered but not used because its projected pattern has lower resolution than the Xtion pattern(Schöps et al., 2019, Supp. Sec. 3.1, Fig. 3)
RGB-D 相機Asus Xtion Live Pro資料集感測器ETH3D SLAM benchmark (this paper)mounted on the rig and used only as infrared pattern emitter for active stereo; its own depth estimation not used(Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1, footnote 1)
RGB-D 相機Kinect v1資料集感測器TUM RGB-Dsensor of the TUM RGB-D benchmark; rolling shutter and unsynchronised depth and colour; its shutter times (about 30.5 ms depth, 26.1 ms colour, from [52]) used to render synthetic rolling-shutter variants; visible depth distortion pattern in BAD SLAM self-calibration(Schöps et al., 2019, Sec. 5 Impact of distortions; Supp. Fig. 9)
運算硬體Intel Core i7 6700K執行運算平台未標示原文未報告(Schöps et al., 2019, Sec. 6 Test environment)
運算硬體MSI Geforce GTX 1080 Gaming X 8G執行運算平台未標示BA implemented on the GPU with CUDA 8.0(Schöps et al., 2019, Sec. 6 Test environment)
其他Vicon motion capturing system參考或真值量測ETH3D SLAM benchmark (this paper)tracks passive markers on the rig; outliers removed manually and trajectory smoothed with an SE(3) cubic B-spline (20 knots per second); camera-Vicon calibration ATE RMSE required to be at most 1 mm(Schöps et al., 2019, Sec. 5; Supp. Sec. 3, 4.4, 4.5, 4.7)

作者報告的優勢與限制

優勢

限制

營建工程相關證據

論文未報告營建測試。其結論指出感測器同步、快門型式與校正誤差會改變方法排名,直接呼應本文對工程點雲誤差來源與公平比較的要求(推論連結)。作者另指出白牆這類弱紋理且幾何變化少的區域會讓面元在表面內任意滑動,因此限制面元只沿法向移動;困難序列的失敗原因包含無紋理且結構模糊的場景(Sec. 3.1、Sec. 6),與室內裝修前的素牆環境相關(推論)。

原文驗證環境:公開基準、受控實驗、模擬、獨立參考量測

報告的性能數據

以下是原文作者報告的性能數值(author-reported results),不是本研究重新量測的結果。每張圖只並列同一個比較組(comparison group,同一張表、同一組實驗設定)內的方法;不同比較組之間的數值不可直接比較,也不構成排名。

本方法共出現在 11 個比較組,合計 46 筆紀錄。以下列出本方法紀錄最多的 4 組,其餘 7 組列在最後,並連到性能比較頁。

Schöps et al., 2019 · Table 3 本方法 8 筆

表格設定(擷取紀錄原文):Synthetic renders of dense TUM RGB-D reconstructions along the original trajectories; each value aggregates ATE RMSE over seven synthetic datasets per category (avg. or med.); rs uses Kinect v1 shutter times (about 30.5 ms depth, 26.1 ms colour); async renders colour midway between depth frames (Schöps et al., 2019, Table 3)

ATE RMSE [cm], average over seven synthetic datasets,synthetic TUM RGB-D renders (7 datasets per category) · clean

只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。

按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。

這些是 Schöps et al., 2019 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。

統計量:平均值(mean);對齊方式:原文未報告;單位:cm;場景:synthetic renders of dense TUM RGB-D scene reconstructions along the original trajectories (Sec. 5)

資料來源作者報告值(Schöps et al., 2019, Table 3)

數值與出處
方法(原文寫法)報告值出處
BundleFusion0.34 cm(Schöps et al., 2019, Table 3)
DVO SLAM0.32 cm(Schöps et al., 2019, Table 3)
ElasticFusion1.11 cm(Schöps et al., 2019, Table 3)
ORB-SLAM20.47 cm(Schöps et al., 2019, Table 3)
BAD SLAM (Ours)本方法原文提出0.15 cm(Schöps et al., 2019, Table 3)

Schöps et al., 2019 · Table 2 本方法 6 筆

指標ATE RMSE [cm]

表格設定(擷取紀錄原文):TUM RGB-D ATE RMSE in cm (rank column omitted); values of other methods copied by the authors from BundleFusion, PSM SLAM and ORB-SLAM2 papers; 'fixed intr.' disables intrinsics and depth-deformation optimisation (Schöps et al., 2019, Table 2)

ATE RMSE [cm],TUM RGB-D · fr1/desk

只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。

按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。

這些是 Schöps et al., 2019 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。

統計量:均方根誤差(RMSE);對齊方式:原文未報告;單位:cm;場景:TUM RGB-D real-world sequences recorded with a Kinect v1 (rolling shutter; depth and colour streams not synchronised, Sec. 5); scene type and carrying mode not described in this paper

資料來源作者報告值(Schöps et al., 2019, Table 2)

數值與出處
方法(原文寫法)報告值出處
BundleFusion1.6 cm(Schöps et al., 2019, Table 2)
DVO SLAM2.1 cm(Schöps et al., 2019, Table 2)
ElasticFusion2 cm(Schöps et al., 2019, Table 2)
Kintinuous3.7 cm(Schöps et al., 2019, Table 2)
MRSMap4.3 cm(Schöps et al., 2019, Table 2)
ORB-SLAM21.6 cm(Schöps et al., 2019, Table 2)
PSM SLAM1.6 cm(Schöps et al., 2019, Table 2)
RGB-D SLAM2.3 cm(Schöps et al., 2019, Table 2)
VoxelHashing2.3 cm(Schöps et al., 2019, Table 2)
BAD SLAM ablation: Ours (fixed intr.)本方法3.6 cm(Schöps et al., 2019, Table 2)
BAD SLAM (Ours)本方法原文提出1.7 cm(Schöps et al., 2019, Table 2)

Sandström et al., 2023 · Table 3 本方法 6 筆

指標ATE RMSE [cm]

表格設定(擷取紀錄原文):TUM-RGBD tracking, ATE RMSE in cm; values in parentheses are averages over successful runs only; N/A not available; ground truth from an external motion capture system (Sandström et al., 2023, Table 3)

ATE RMSE [cm],TUM-RGBD · fr1/desk

只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。

按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。

這些是 Sandström et al., 2023 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。

統計量:均方根誤差(RMSE);對齊方式:原文未報告;單位:cm;場景:indoor office (real RGB-D)

資料來源作者報告值(Sandström et al., 2023, Table 3)

數值與出處
方法(原文寫法)報告值出處
DI-Fusion [19]4.4 cm(Sandström et al., 2023, Table 3)
NICE-SLAM [81]4.26 cm(Sandström et al., 2023, Table 3)
Vox-Fusion* [71] (re-run)3.52 cm(Sandström et al., 2023, Table 3)
Point-SLAM (ours)原文提出4.34 cm(Sandström et al., 2023, Table 3)
BAD-SLAM [50]本方法1.7 cm(Sandström et al., 2023, Table 3)
Kintinuous [68]3.7 cm(Sandström et al., 2023, Table 3)
ORB-SLAM2 [35]1.6 cm(Sandström et al., 2023, Table 3)
ElasticFusion [67]2.53 cm(Sandström et al., 2023, Table 3)

Yan et al., 2024 · Table 2 本方法 4 筆

指標ATE [cm] (text calls it ATE RSME)

表格設定(擷取紀錄原文):TUM RGB-D ATE on three sequences; * = reproduced with official code (Yan et al., 2024, Table 2)

ATE [cm] (text calls it ATE RSME),TUM RGB-D · fr1_desk

只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。

按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。

這些是 Yan et al., 2024 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。

統計量:均方根誤差(RMSE);對齊方式:原文未報告;單位:cm;場景:real indoor (office, desk)

資料來源作者報告值(Yan et al., 2024, Table 2)

數值與出處
方法(原文寫法)報告值出處
DI-Fusion [ 9 ]4.4 cm(Yan et al., 2024, Table 2)
ElasticFusion [ 46 ]2.5 cm(Yan et al., 2024, Table 2)
BAD-SLAM [ 30 ]本方法1.7 cm(Yan et al., 2024, Table 2)
Kintinuous [ 45 ]3.7 cm(Yan et al., 2024, Table 2)
ORB-SLAM2 [ 20 ]1.6 cm(Yan et al., 2024, Table 2)
iMAP ∗ [ 35 ]7.2 cm(Yan et al., 2024, Table 2)
NICE-SLAM [ 55 ]4.3 cm(Yan et al., 2024, Table 2)
Vox-Fusion ∗ [ 48 ]3.5 cm(Yan et al., 2024, Table 2)
CoSLAM [ 41 ]2.7 cm(Yan et al., 2024, Table 2)
ESLAM [ 11 ]2.3 cm(Yan et al., 2024, Table 2)
Point-SLAM2.6 cm(Yan et al., 2024, Table 2)
Ours原文提出3.3 cm(Yan et al., 2024, Table 2)

其他比較組

列出其餘 7 個比較組

來源

  • Schöps et al., 2019

    Thomas Schöps, Torsten Sattler, Marc Pollefeys(2019)BAD SLAM: Bundle Adjusted Direct RGB-D SLAM2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 134-144

    同儕審查已出版已讀全文近十年查證後修正

回到方法圖鑑

選擇開啟Esc關閉