BAD SLAM
BAD SLAM 提出可即時執行的直接式 BA,以面元表示地圖,同時使用深度的幾何約束與影像梯度的光度約束,並交替最佳化地圖與相機位姿。作者另建立以同步全域快門(global shutter)RGB 與深度相機錄製、經精確校正的 ETH3D SLAM 基準,指出直接式 RGB-D SLAM 對捲簾快門、RGB 與深度不同步及校正誤差高度敏感。在此基準上方法排名與既有資料集不同,顯示資料集設定本身會左右比較結論。
本頁內容
BAD SLAM performs real-time direct bundle adjustment over surfels and keyframes and introduces a well-calibrated, synchronised global-shutter RGB-D benchmark, showing direct RGB-D SLAM is highly sensitive to rolling shutter, RGB-depth synchronisation and calibration errors.
技術屬性
欄位內容為文獻擷取紀錄的原文用語(英文),以原文為據;「未查證」表示本研究尚未讀到該資訊,不代表該方法不具備此能力。
| 感測輸入 | RGB-D |
|---|---|
| 原文測試平台 | custom multi-camera rig with up to eight synchronized global-shutter cameras (two colour and two infrared cameras form the RGB-D sensor; four more used only for SfM ground truth) and an Asus Xtion Live Pro used solely as infrared pattern emitter, tracked by a Vicon system; carrying mode not stated in paper or supplement、public TUM RGB-D sequences (Kinect v1)、synthetic renders of TUM RGB-D reconstructions |
| 狀態估計 | Alternating direct BA (Alg. 1): Gauss-Newton on a cost of Tukey-weighted point-to-plane residuals and Huber-weighted photometric gradient residuals (photometric weight 1e-2), each normalised by a stereo depth-noise model or an empirical sigma; surfels move only along their normals (joint 2x2 solve of offset and descriptor per surfel); independent per-keyframe SE(3) pose updates; optional intrinsics and per-pixel depth-deformation calibration solved with the Schur complement; interleaved discrete surfel creation, merging, outlier deletion and radius update. A PCG Gauss-Newton solver was slightly worse in the ablation. |
| 資料關聯 | Surfel to pixel correspondences by projecting surfel centres into every keyframe, kept only if the pixel has depth, the Tukey weight of the geometric residual is positive and normals agree; photometric residual compares the surfel descriptor with the gradient magnitude sampled at the surfel centre and two disc boundary points. Front-end odometry: direct photometric and geometric SE(3) alignment of each frame to the last keyframe using intensity gradients. |
| 時間表示 | discrete poses; every 10th frame becomes a keyframe; benchmark colour and depth are captured at the same instants, so no temporal interpolation between them is needed |
| 去畸變 | 不適用 (rolling shutter avoided by global-shutter, synchronised benchmark cameras rather than modelled) |
| 迴圈閉合 | bag-of-words detection with binary features; relative pose from keypoint matches refined by direct alignment and checked for consistency against neighbouring keyframes m-1 and m+1; then pose-graph optimisation (Sec. 4; Fig. 2) |
| 全域最佳化 | pose-graph optimisation on loop detection, plus direct BA back-end over keyframe poses, surfels and camera intrinsics (Fig. 2; Sec. 3-4) |
| 地圖表示 | surfels (oriented discs with centre, normal, radius and a scalar gradient descriptor) created on 4x4 pixel cells of each keyframe, plus keyframes storing raw RGB-D; about 335,000 surfels for the Fig. 1 scene |
| 先驗資訊 | none |
| 可輸出幾何 | surfel map and keyframe trajectory |
| 計算需求 | Intel Core i7 6700K with MSI GeForce GTX 1080 Gaming X 8G; BA on GPU with CUDA 8.0; with about 27 Hz input and one keyframe per 10 frames, 370 ms of BA time per keyframe are available and further BA iterations are skipped in real-time mode; odometry time negligible; memory grows linearly with keyframe count |
使用設備
原文使用的感測器、運算硬體與載具(equipment)。型號保留原文寫法,連結到設備頁中同一型號的歸併名稱;角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測(reference or ground truth)與比較對象設備。
| 類別 | 型號(原文寫法) | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| 慣性量測單元(IMU) | IMU (model not stated in paper or supplement) | 資料集感測器 | ETH3D SLAM benchmark (this paper) | benchmark stated to provide full IMU data | (Schöps et al., 2019, Sec. 2, Table 1) |
| 相機 | four additional synchronized cameras on the rig | 參考或真值量測 | ETH3D SLAM benchmark (this paper) | used only to localise training sequences whose ground truth comes from Structure-from-Motion | (Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1) |
| 雙目相機 | two synchronized global-shutter colour cameras (front-facing stereo pair) on a custom multi-camera rig similar to Gohl et al. [22] | 資料集感測器 | ETH3D SLAM benchmark (this paper) | one camera supplies the RGB image of the RGB-D frames; both usable for stereo SLAM; raw Bayer images debayered, flat-field corrected, no white balancing | (Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1, 4.1, Fig. 2) |
| 雙目相機 | two synchronized global-shutter infrared cameras (stereo pair below the colour cameras) on the same rig | 資料集感測器 | ETH3D SLAM benchmark (this paper) | active stereo depth by PatchMatch stereo with ZNCC cost and 11x11 window, reprojected to the colour camera; colour and depth recorded at exactly the same time | (Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1, 4.6) |
| 雙目相機 | Intel D435 | 比較對象設備 | 未標示 | infrared emitter considered but not used because its projected pattern has lower resolution than the Xtion pattern | (Schöps et al., 2019, Supp. Sec. 3.1, Fig. 3) |
| RGB-D 相機 | Asus Xtion Live Pro | 資料集感測器 | ETH3D SLAM benchmark (this paper) | mounted on the rig and used only as infrared pattern emitter for active stereo; its own depth estimation not used | (Schöps et al., 2019, Sec. 5; Supp. Sec. 3.1, footnote 1) |
| RGB-D 相機 | Kinect v1 | 資料集感測器 | TUM RGB-D | sensor of the TUM RGB-D benchmark; rolling shutter and unsynchronised depth and colour; its shutter times (about 30.5 ms depth, 26.1 ms colour, from [52]) used to render synthetic rolling-shutter variants; visible depth distortion pattern in BAD SLAM self-calibration | (Schöps et al., 2019, Sec. 5 Impact of distortions; Supp. Fig. 9) |
| 運算硬體 | Intel Core i7 6700K | 執行運算平台 | 未標示 | 原文未報告 | (Schöps et al., 2019, Sec. 6 Test environment) |
| 運算硬體 | MSI Geforce GTX 1080 Gaming X 8G | 執行運算平台 | 未標示 | BA implemented on the GPU with CUDA 8.0 | (Schöps et al., 2019, Sec. 6 Test environment) |
| 其他 | Vicon motion capturing system | 參考或真值量測 | ETH3D SLAM benchmark (this paper) | tracks passive markers on the rig; outliers removed manually and trajectory smoothed with an SE(3) cubic B-spline (20 knots per second); camera-Vicon calibration ATE RMSE required to be at most 1 mm | (Schöps et al., 2019, Sec. 5; Supp. Sec. 3, 4.4, 4.5, 4.7) |
作者報告的優勢與限制
優勢
- Direct BA with rich information yields very accurate trajectories on the new benchmark (conclusion; Figs. 6 and 7)
- Open benchmark with hidden test set and online evaluation (abstract)
- Lowest average and median ATE RMSE of the five systems on all four synthetic distortion variants (Table 3)
- Rankings on the benchmark do not depend on using ATE or relative translation and rotation metrics (Supp. Sec. 2.1)
限制
- Direct RGB-D SLAM is highly sensitive to rolling shutter, RGB-depth synchronisation and calibration errors (abstract; Table 3)
- Benchmark contains hard sequences that remain open challenges for visual-only RGB-D SLAM (conclusion)
- On TUM RGB-D ORB-SLAM2 outperforms all direct methods including BAD SLAM, and without intrinsics and depth-deformation optimisation BAD SLAM is clearly worse (Table 2)
- Direct alignment has a small convergence region; a pose-graph step after loop detection could in principle push old keyframes out of it (Sec. 4)
- Failures come from textureless scenes with ambiguous structure, fast motion and moving objects; an IMU or wider field of view would be needed (Sec. 6)
- Scalability not evaluated; real-time BA for longer sequences would need e.g. windowed BA (Sec. 6, Sec. 7)
營建工程相關證據
論文未報告營建測試。其結論指出感測器同步、快門型式與校正誤差會改變方法排名,直接呼應本文對工程點雲誤差來源與公平比較的要求(推論連結)。作者另指出白牆這類弱紋理且幾何變化少的區域會讓面元在表面內任意滑動,因此限制面元只沿法向移動;困難序列的失敗原因包含無紋理且結構模糊的場景(Sec. 3.1、Sec. 6),與室內裝修前的素牆環境相關(推論)。
原文驗證環境:公開基準、受控實驗、模擬、獨立參考量測
報告的性能數據
以下是原文作者報告的性能數值(author-reported results),不是本研究重新量測的結果。每張圖只並列同一個比較組(comparison group,同一張表、同一組實驗設定)內的方法;不同比較組之間的數值不可直接比較,也不構成排名。
本方法共出現在 11 個比較組,合計 46 筆紀錄。以下列出本方法紀錄最多的 4 組,其餘 7 組列在最後,並連到性能比較頁。
Schöps et al., 2019 · Table 3 本方法 8 筆
表格設定(擷取紀錄原文):Synthetic renders of dense TUM RGB-D reconstructions along the original trajectories; each value aggregates ATE RMSE over seven synthetic datasets per category (avg. or med.); rs uses Kinect v1 shutter times (about 30.5 ms depth, 26.1 ms colour); async renders colour midway between depth frames (Schöps et al., 2019, Table 3)
ATE RMSE [cm], average over seven synthetic datasets,synthetic TUM RGB-D renders (7 datasets per category) · clean
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Schöps et al., 2019 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Schöps et al., 2019, Table 3)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| BundleFusion | 0.34 cm | (Schöps et al., 2019, Table 3) |
| DVO SLAM | 0.32 cm | (Schöps et al., 2019, Table 3) |
| ElasticFusion | 1.11 cm | (Schöps et al., 2019, Table 3) |
| ORB-SLAM2 | 0.47 cm | (Schöps et al., 2019, Table 3) |
| BAD SLAM (Ours)本方法原文提出 | 0.15 cm | (Schöps et al., 2019, Table 3) |
Schöps et al., 2019 · Table 2 本方法 6 筆
指標ATE RMSE [cm]
表格設定(擷取紀錄原文):TUM RGB-D ATE RMSE in cm (rank column omitted); values of other methods copied by the authors from BundleFusion, PSM SLAM and ORB-SLAM2 papers; 'fixed intr.' disables intrinsics and depth-deformation optimisation (Schöps et al., 2019, Table 2)
ATE RMSE [cm],TUM RGB-D · fr1/desk
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Schöps et al., 2019 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Schöps et al., 2019, Table 2)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| BundleFusion | 1.6 cm | (Schöps et al., 2019, Table 2) |
| DVO SLAM | 2.1 cm | (Schöps et al., 2019, Table 2) |
| ElasticFusion | 2 cm | (Schöps et al., 2019, Table 2) |
| Kintinuous | 3.7 cm | (Schöps et al., 2019, Table 2) |
| MRSMap | 4.3 cm | (Schöps et al., 2019, Table 2) |
| ORB-SLAM2 | 1.6 cm | (Schöps et al., 2019, Table 2) |
| PSM SLAM | 1.6 cm | (Schöps et al., 2019, Table 2) |
| RGB-D SLAM | 2.3 cm | (Schöps et al., 2019, Table 2) |
| VoxelHashing | 2.3 cm | (Schöps et al., 2019, Table 2) |
| BAD SLAM ablation: Ours (fixed intr.)本方法 | 3.6 cm | (Schöps et al., 2019, Table 2) |
| BAD SLAM (Ours)本方法原文提出 | 1.7 cm | (Schöps et al., 2019, Table 2) |
Sandström et al., 2023 · Table 3 本方法 6 筆
指標ATE RMSE [cm]
表格設定(擷取紀錄原文):TUM-RGBD tracking, ATE RMSE in cm; values in parentheses are averages over successful runs only; N/A not available; ground truth from an external motion capture system (Sandström et al., 2023, Table 3)
ATE RMSE [cm],TUM-RGBD · fr1/desk
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Sandström et al., 2023 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Sandström et al., 2023, Table 3)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| DI-Fusion [19] | 4.4 cm | (Sandström et al., 2023, Table 3) |
| NICE-SLAM [81] | 4.26 cm | (Sandström et al., 2023, Table 3) |
| Vox-Fusion* [71] (re-run) | 3.52 cm | (Sandström et al., 2023, Table 3) |
| Point-SLAM (ours)原文提出 | 4.34 cm | (Sandström et al., 2023, Table 3) |
| BAD-SLAM [50]本方法 | 1.7 cm | (Sandström et al., 2023, Table 3) |
| Kintinuous [68] | 3.7 cm | (Sandström et al., 2023, Table 3) |
| ORB-SLAM2 [35] | 1.6 cm | (Sandström et al., 2023, Table 3) |
| ElasticFusion [67] | 2.53 cm | (Sandström et al., 2023, Table 3) |
Yan et al., 2024 · Table 2 本方法 4 筆
指標ATE [cm] (text calls it ATE RSME)
表格設定(擷取紀錄原文):TUM RGB-D ATE on three sequences; * = reproduced with official code (Yan et al., 2024, Table 2)
ATE [cm] (text calls it ATE RSME),TUM RGB-D · fr1_desk
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Yan et al., 2024 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Yan et al., 2024, Table 2)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| DI-Fusion [ 9 ] | 4.4 cm | (Yan et al., 2024, Table 2) |
| ElasticFusion [ 46 ] | 2.5 cm | (Yan et al., 2024, Table 2) |
| BAD-SLAM [ 30 ]本方法 | 1.7 cm | (Yan et al., 2024, Table 2) |
| Kintinuous [ 45 ] | 3.7 cm | (Yan et al., 2024, Table 2) |
| ORB-SLAM2 [ 20 ] | 1.6 cm | (Yan et al., 2024, Table 2) |
| iMAP ∗ [ 35 ] | 7.2 cm | (Yan et al., 2024, Table 2) |
| NICE-SLAM [ 55 ] | 4.3 cm | (Yan et al., 2024, Table 2) |
| Vox-Fusion ∗ [ 48 ] | 3.5 cm | (Yan et al., 2024, Table 2) |
| CoSLAM [ 41 ] | 2.7 cm | (Yan et al., 2024, Table 2) |
| ESLAM [ 11 ] | 2.3 cm | (Yan et al., 2024, Table 2) |
| Point-SLAM | 2.6 cm | (Yan et al., 2024, Table 2) |
| Ours原文提出 | 3.3 cm | (Yan et al., 2024, Table 2) |
其他比較組
來源
Schöps et al., 2019
(2019)BAD SLAM: Bundle Adjusted Direct RGB-D SLAM2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 134-144
DOI 10.1109/cvpr.2019.00022程式碼
同儕審查已出版已讀全文近十年查證後修正
相關版本
- 程式碼釋出:badslam https://github.com/ETH3D/badslam
程式碼:https://github.com/ETH3D/badslam(授權:BSD-style (LICENSE header, ETH Zurich 2019))。有公開程式碼不等於已被重現,也不代表目前版本與論文版本相同。