MRSMap
MRSMap 把每張 RGB-D 影像轉成八元樹多解析度面元地圖:各層節點都以單次掃描累加的充分統計量,保存點位置與 Lαβ 色彩的六維常態分布,並依最多六個觀測方向分開保存面元;最細解析度隨深度平方放寬,以反映 RGB-D 深度雜訊。配準時由最細解析度開始,在鄰域中尋找形狀與紋理描述子相符的面元配對,先以 Levenberg-Marquardt 初始化、再以牛頓法最大化配對似然,並以閉式近似估計位姿共變異。SLAM 以關鍵視角為節點,每一影格隨機抽選一個鄰近關鍵視角嘗試配準以發現迴圈,再以 g2o 最佳化位姿圖,全程在 CPU 上即時執行,同一方法也用於物件建模與追蹤。
本頁內容
CPU RGB-D registration and key-view SLAM on octree multi-resolution surfel maps: each node stores a 6D position-colour Gaussian per view direction with depth-adapted finest resolution; maps are registered by descriptor-gated multi-resolution surfel association with LM then Newton optimization and closed-form pose covariance, and key views are linked by randomized loop-closure tests and g2o pose-graph optimization.
技術屬性
欄位內容為文獻擷取紀錄的原文用語(英文),以原文為據;「未查證」表示本研究尚未讀到該資訊,不代表該方法不具備此能力。
| 感測輸入 | RGB-D camera at VGA 640x480 and 30 Hz (TUM Freiburg benchmark sequences and the authors' object dataset; sensor model not named); QVGA used on the robot |
|---|---|
| 原文測試平台 | RGB-D camera moved through the TUM Freiburg scenes and around objects for the authors' object dataset (carrying mode not stated, Secs. 5 and 6)、mobile manipulation robot Cosero (RoboCup@Home 2011 and 2012 demonstrations; locomotion not described, Sec. 6.4) |
| 狀態估計 | Key-view SLAM: each frame is registered to the current reference key view by maximizing the surfel-match likelihood (approximate Levenberg-Marquardt initialization, then Newton's method with trilinear interpolation, typically 10 to 20 LM and 5 Newton iterations); key views linked by relative-pose constraints with closed-form covariance; pose graph solved by sparse Cholesky in g2o, one iteration per frame |
| 資料關聯 | Multi-resolution surfel association starting at the finest resolution with a cubic volume query around the transformed surfel mean (side twice the node resolution), bootstrapped from previous associations via the 26-neighbourhood; accepted only if shape-texture descriptors (surfel-pair angle histograms and luminance and chrominance contrasts) differ by at most 0.1 and contour flags agree |
| 時間表示 | discrete poses |
| 去畸變 | 不適用 (RGB-D input) |
| 迴圈閉合 | Randomized hypothesis-and-test: per frame one key view, sampled with probability decreasing with distance and angle from the reference, is registered; the constraint is accepted if its bidirectional matching likelihood is at least a fraction of that of the key view's initial constraint |
| 全域最佳化 | Pose graph over key views in g2o, iterated once per frame (median 0.79 ms, max 4.01 ms on freiburg2_desk with 64 key views and 138 edges); optimized key views fused into one multi-view map |
| 地圖表示 | octree multi-resolution surfel map: every node stores sufficient statistics of a 6D Gaussian of position and L-alpha-beta colour, up to six surfels per node for orthogonal view directions; finest node size adapted to squared depth (0.0125 m limit); border and occluded-background surfels excluded |
| 先驗資訊 | none |
| 可輸出幾何 | multi-view multi-resolution surfel map of a scene or an object model (visualized by sampling the surfel distributions; object models about 54 MB for a chair and 19 MB for a humanoid) |
| 計算需求 | CPU only (multi-core parallel association and derivatives); notebook Intel Core i7 3610QM 2.3 GHz; registration about 15 Hz at VGA (61 to 75 ms per frame); real-time SLAM drops frames and limits maximum resolution to 0.05 m |
使用設備
原文使用的感測器、運算硬體與載具(equipment)。型號保留原文寫法,連結到設備頁中同一型號的歸併名稱;角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測(reference or ground truth)與比較對象設備。
| 類別 | 型號(原文寫法) | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| 載具平台 | Cosero mobile manipulation robot | 方法輸入 | 未標示 | used the tracking method at RoboCup@Home 2011 and 2012 | (Stückler & Behnke, 2014, Sec. 6.4) |
| 運算硬體 | notebook PC with Intel Core i7 3610QM 2.3 GHz (max. 3.3 GHz) QuadCore CPU | 執行運算平台 | 未標示 | timings of all methods at VGA resolution | (Stückler & Behnke, 2014, Sec. 6) |
| 運算硬體 | quadcore notebook with Intel i7-Q720 CPU (Cosero's main computer) | 執行運算平台 | 未標示 | RGB-D images subsampled to QVGA for tracking during RoboCup@Home demonstrations | (Stückler & Behnke, 2014, Sec. 6.4) |
| 其他 | external optical motion capture systems (models not reported) | 參考或真值量測 | TUM RGB-D and authors' object tracking dataset | ground-truth camera poses of the TUM benchmark and the authors' object dataset | (Stückler & Behnke, 2014, Sec. 6) |
作者報告的優勢與限制
優勢
- Lowest median incremental-registration RPE on most of 22 TUM sequences against warp, GICP, 3D-NDT and fovis (Table 1)
- About 15 Hz at VGA on a notebook CPU: 75 ms versus 4015 ms for GICP and 415 ms for 3D-NDT on fr1 desk (Table 2)
- Keeps ICP-like robustness for larger frame skips where warp fails (Fig. 6)
- Object models with median ATE of about 1 to 2 cm, and graph optimization strongly reduces error (Table 4)
- Used for real-time object tracking on the Cosero robot at RoboCup@Home 2011 and 2012 (Sec. 6.4)
限制
- Did not succeed on freiburg1_floor (little texture) or freiburg2_large_loop, and loses track when much of the image has invalid or highly uncertain distant depth (Sec. 6.2)
- Real-time SLAM must drop frames and limit the maximum resolution to 0.05 m (Sec. 6.2)
- Loop detection relies on pose proximity; appearance-based loop detection for larger scenes is future work (Sec. 5.1.1, Sec. 7)
- Fovis is much faster (13 to 16 ms) and more accurate on some textured sequences (Tables 1 and 2)
- The SLAM RPE values of Table 3 were corrected by a 2015 corrigendum; with the corrected values the method beats RGB-D SLAM on 8 of 11 sequences rather than the nine stated in Sec. 6.2 (inference from the corrigendum)
營建工程相關證據
論文未在施工現場測試,評估為 TUM (Freiburg) 室內小場景與桌上物件資料集,並在 RoboCup@Home 競賽中讓服務機器人即時追蹤桌子與鍋具。在 CPU 上以多解析度面元配準 RGB-D 影像,適合運算資源有限的室內機器人;但論文 Table 3 的 SLAM 精度已由 2015 年更正啟事(JVCIR 26:349)修訂,引用時應採更正後數值。corpus 中 Dai et al., 2017a、Whelan et al., 2015a、Whelan et al., 2015b 與 Schöps et al., 2019 皆以 MRSMap 作為比較基準。
原文驗證環境:公開基準、受控實驗、任務層驗證
報告的性能數據
以下是原文作者報告的性能數值(author-reported results),不是本研究重新量測的結果。每張圖只並列同一個比較組(comparison group,同一張表、同一組實驗設定)內的方法;不同比較組之間的數值不可直接比較,也不構成排名。
本方法共出現在 7 個比較組,合計 60 筆紀錄。以下列出本方法紀錄最多的 4 組,其餘 3 組列在最後,並連到性能比較頁。
Stückler & Behnke, 2014 · Table 1 本方法 22 筆
指標median relative pose error (RPE) in mm
表格設定(擷取紀錄原文):Incremental (frame-to-frame) registration on TUM Freiburg sequences; median translational relative pose error in mm (maximum values in brackets in the table not extracted); warp is the OpenCV reimplementation (Stückler & Behnke, 2014, Table 1)
median relative pose error (RPE) in mm,TUM RGB-D (Freiburg) · fr1 360
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Stückler & Behnke, 2014 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Stückler & Behnke, 2014, Table 1)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Ours (MRSMap)本方法原文提出 | 5.1 mm | (Stückler & Behnke, 2014, Table 1) |
| Warp [17] (OpenCV) | 5.9 mm | (Stückler & Behnke, 2014, Table 1) |
| GICP [5] | 18.8 mm | (Stückler & Behnke, 2014, Table 1) |
| 3D-NDT [7] | 7.8 mm | (Stückler & Behnke, 2014, Table 1) |
| Fovis [12] | 7.1 mm | (Stückler & Behnke, 2014, Table 1) |
Stückler & Behnke, 2014 · Table 3 (corrigendum) 本方法 22 筆
指標RMSE RPE in m
表格設定(擷取紀錄原文):SLAM on TUM Freiburg sequences: RMSE of relative pose error averaged over all frame differences, in m; values from the 2015 corrigendum that replaces the originally printed Table 3 (Stückler & Behnke, 2014, Table 3 (corrigendum))
RMSE RPE in m,TUM RGB-D (Freiburg) · freiburg1_360
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Stückler & Behnke, 2014 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Stückler & Behnke, 2014, Table 3 (corrigendum))
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Ours all frames (MRSMap SLAM)本方法原文提出 | 0.123 m | (Stückler & Behnke, 2014, Corrigendum Table 3 (replaces article Table 3)) |
| Ours real-time (MRSMap SLAM, frames dropped, 0.05 m max. resolution)本方法原文提出 | 0.126 m | (Stückler & Behnke, 2014, Corrigendum Table 3 (replaces article Table 3)) |
| RGB-D SLAM [24], [34] (Endres et al.) | 0.103 m | (Stückler & Behnke, 2014, Corrigendum Table 3 (replaces article Table 3)) |
Yan et al., 2017 · Table 2 本方法 4 筆
指標ATE [m]
表格設定(擷取紀錄原文):Complete SLAM absolute trajectory error on TUM RGB-D; '-' = no result given (Yan et al., 2017, Table 2)
ATE [m],TUM RGB-D · fr1/desk
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Yan et al., 2017 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Yan et al., 2017, Table 2)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| RGB-D SLAM [4] (Endres et al.) | 0.023 m | (Yan et al., 2017, Table 2) |
| Kintinuous [33] | 0.037 m | (Yan et al., 2017, Table 2) |
| MRSMap [28]本方法 | 0.043 m | (Yan et al., 2017, Table 2) |
| ElasticFusion [34] | 0.02 m | (Yan et al., 2017, Table 2) |
| DVO SLAM [14] | 0.021 m | (Yan et al., 2017, Table 2) |
| sigma-DVO SLAM | 0.019 m | (Yan et al., 2017, Table 2) |
| PSM SLAM原文提出 | 0.016 m | (Yan et al., 2017, Table 2) |
Yan et al., 2017 · Table 4 本方法 4 筆
指標mean distance from points to nearest ground-truth surface (m)
表格設定(擷取紀錄原文):Surface reconstruction accuracy on ICL-NUIM living room with noise: mean distance from reconstructed points to the nearest ground-truth surface (m) (Yan et al., 2017, Table 4)
mean distance from points to nearest ground-truth surface (m),ICL-NUIM · lr kt0
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Yan et al., 2017 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Yan et al., 2017, Table 4)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| RGB-D SLAM [4] (Endres et al.) | 0.044 m | (Yan et al., 2017, Table 4) |
| Kintinuous [33] | 0.011 m | (Yan et al., 2017, Table 4) |
| MRSMap [28]本方法 | 0.061 m | (Yan et al., 2017, Table 4) |
| DVO SLAM [14] | 0.032 m | (Yan et al., 2017, Table 4) |
| ElasticFusion [34] | 0.007 m | (Yan et al., 2017, Table 4) |
| PSM SLAM原文提出 | 0.006 m | (Yan et al., 2017, Table 4) |
其他比較組
來源
Stückler & Behnke, 2014
(2014)Multi-resolution surfel maps for efficient dense 3D modeling and trackingJournal of Visual Communication and Image Representation, 25(1):137-147
DOI 10.1016/j.jvcir.2013.02.008程式碼
同儕審查已出版已讀全文經典查證後修正
相關版本
- 勘誤或補遺:Corrigendum to 'Multi-resolution surfel maps for efficient dense 3D modeling and tracking', JVCIR 26 (2015) 349, which replaces Table 3 10.1016/j.jvcir.2014.10.001
- 程式碼釋出:jstueckler/mrsmap (original release at code.google.com/p/mrsmap) https://github.com/jstueckler/mrsmap
程式碼:https://github.com/jstueckler/mrsmap(授權:BSD licence (LICENSE file checked, University of Bonn))。有公開程式碼不等於已被重現,也不代表目前版本與論文版本相同。