StaticFusion
StaticFusion 是針對動態環境的 RGB-D 稠密 SLAM,同時估計相機運動與影像中哪些區域靜止。每張影像先以 K-means 依三維座標分成幾何群集,再與由靜態面元地圖渲染出的預測影像做光度與幾何直接對齊;每個群集有一個 0 至 1 的靜態分數,用來加權其殘差,分數則由殘差門檻、相鄰群集平滑與深度差先驗共同決定,並以 IRLS 交替求解位姿與分數。融合時只寫入靜態部分,每個面元以對數勝算累積被靜態點重複觀測的可信度,持續被動態點匹配的面元會被移除,因此地圖只保留背景結構,並把這個背景作為下一影格分割所需的時間資訊。
本頁內容
Dense RGB-D SLAM for dynamic scenes that jointly estimates camera motion and per-cluster static scores by score-weighted photometric and geometric alignment against a rendered static surfel map, and fuses only static data with per-surfel viability so the map keeps only the background.
技術屬性
欄位內容為文獻擷取紀錄的原文用語(英文),以原文為據;「未查證」表示本研究尚未讀到該資訊,不代表該方法不具備此能力。
| 感測輸入 | RGB-D camera, registered RGB-D images at QVGA 320x240 (TUM Freiburg sequences and two hand-held recordings; hand-held camera model not named) |
|---|---|
| 原文測試平台 | hand-held RGB-D camera (two authors' sequences, including a 'selfie' sequence, Sec. VII-B)、TUM Freiburg benchmark sequences (capture platform not described in the paper) |
| 狀態估計 | Joint minimization over the camera twist and per-cluster static scores b in [0, 1]: Cauchy-robust photometric and geometric residuals between the current RGB-D frame and a prediction rendered from the static surfel map, weighted by b, plus a residual-threshold term, spatial regularization between contiguous clusters and a depth-difference prior; IRLS for the twist with closed-form b after each iteration, coarse-to-fine |
| 資料關聯 | Dense direct alignment (warping current pixels into the rendered model prediction); scene split into K geometric clusters by K-means on 3D coordinates; per-pixel segmentation derived from cluster scores |
| 時間表示 | discrete poses |
| 去畸變 | 不適用 (RGB-D input) |
| 迴圈閉合 | none described |
| 全域最佳化 | none described |
| 地圖表示 | surfel map (Keller et al. model through the ElasticFusion implementation) in which each surfel carries a viability value accumulated as log-odds of matches with static input points; surfels with viability below 0.5 for more than 10 consecutive frames are removed and free-space violations are cleaned |
| 先驗資訊 | none |
| 可輸出幾何 | static-background surfel map (coloured for visualization), camera trajectory and per-frame static/dynamic segmentation |
| 計算需求 | Intel Core i7-3770 at 3.40 GHz with GeForce GTX 1070 GPU; about 30 ms per frame at QVGA (Sec. VII, VIII) |
使用設備
原文使用的感測器、運算硬體與載具(equipment)。型號保留原文寫法,連結到設備頁中同一型號的歸併名稱;角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測(reference or ground truth)與比較對象設備。
| 類別 | 型號(原文寫法) | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| RGB-D 相機 | hand-held RGB-D camera (model not reported) | 方法輸入 | 未標示 | two recorded sequences: person interacting with objects, and a selfie sequence with the camera pointing at its carrier | (Scona et al., 2018, Sec. VII-B) |
| 運算硬體 | workstation with Intel(R) Core(TM) i7-3770 CPU at 3.40GHz and GeForce GTX 1070 GPU | 執行運算平台 | 未標示 | Ubuntu 16.04; all compared methods run on it | (Scona et al., 2018, Sec. VII) |
作者報告的優勢與限制
優勢
- Lowest ATE on the highly dynamic TUM walking sequences, for example 1.4 cm on fr3/walk_static and 12.7 cm on fr3/walk_xyz versus 29.3 cm and 90.6 cm for ElasticFusion (Table II)
- Accuracy comparable to ElasticFusion in static sequences (Tables I and II; Sec. VII-A)
- In a 9.5 m hand-held 'selfie' sequence the final drift was 1.5 cm versus 1.03 m for ElasticFusion and 0.88 m for Co-Fusion (Sec. VII-B)
- About 30 ms per frame (Sec. VIII)
限制
- Initialization needs the first 1 to 2 s to contain no more than about 20 to 30% moving elements; the method has high error (39.1 cm ATE) on fr3/walk_halfsphere, whose start is highly dynamic (Sec. VI-A, VII-A, VIII)
- Struggles if significant portions of the estimated static scene begin to move (Sec. VIII)
- Static/dynamic segmentation is solved per cluster rather than per pixel, an approximation (Sec. III)
- Less accurate than ElasticFusion on some static sequences, for example fr1/plant ATE 11.3 cm versus 5.3 cm (Table II)
- Evaluated with a desktop GPU (GTX 1070) on a GPU-based ElasticFusion code base (Sec. V, VII)
營建工程相關證據
論文未在施工現場測試,評估為 TUM (Freiburg) 室內序列與兩段手持錄製的室內序列(人員與物件移動、自拍式移動)。只把靜態背景融入地圖並持續移除被動態點匹配的面元,對有人員走動的施工中室內掃描有參考價值;但作者指出初始 1 至 2 秒若動態元素超過約 20% 至 30% 便可能失敗,因此工地掃描起始時宜避開人員密集的畫面(推論)。
原文驗證環境:公開基準、受控實驗
報告的性能數據
以下是原文作者報告的性能數值(author-reported results),不是本研究重新量測的結果。每張圖只並列同一個比較組(comparison group,同一張表、同一組實驗設定)內的方法;不同比較組之間的數值不可直接比較,也不構成排名。
本方法共出現在 6 個比較組,合計 65 筆紀錄。以下列出本方法紀錄最多的 4 組,其餘 2 組列在最後,並連到性能比較頁。
Palazzolo et al., 2019 · Table III 本方法 24 筆
指標Absolute Trajectory Error (RMS) [m]
表格設定(擷取紀錄原文):Bonn RGB-D Dynamic Dataset (24 highly dynamic scenes, ASUS Xtion Pro LIVE, OptiTrack Prime 13 ground truth); ATE RMS; 'o box' = obstructing box, 'no box' = nonobstructing box; all methods run by the authors with default parameters (Palazzolo et al., 2019, Table III)
Absolute Trajectory Error (RMS) [m],Bonn RGB-D Dynamic Dataset · balloon
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Palazzolo et al., 2019 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Palazzolo et al., 2019, Table III)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Ours (ReFusion)原文提出 | 0.175 m | (Palazzolo et al., 2019, Table III) |
| SF (StaticFusion)本方法 | 0.233 m | (Palazzolo et al., 2019, Table III) |
| DS (G) (DynaSLAM geometric) | 0.05 m | (Palazzolo et al., 2019, Table III) |
| DS (N+G) (DynaSLAM neural network + geometric) | 0.03 m | (Palazzolo et al., 2019, Table III) |
Scona et al., 2018 · Table I 本方法 22 筆
表格設定(擷取紀錄原文):TUM (Freiburg) RGB-D sequences grouped as static (fr1), low dynamic (fr3/sit) and high dynamic (fr3/walk) environments; StaticFusion and VO-SF at QVGA, ElasticFusion and Co-Fusion at their default VGA; fr3/walk_halfsphere* skips the first 5 s of high dynamics; relative pose error per second (Scona et al., 2018, Table I)
Trans. RPE RMSE (cm/s),TUM RGB-D (Freiburg) · fr1/xyz
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
- 未報告(沒有數值,不是 0)
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Scona et al., 2018 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Scona et al., 2018, Table I)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| VO-SF (Jaimez et al. joint visual odometry and scene flow) | 2.1 cm/s | (Scona et al., 2018, Table I) |
| EF (ElasticFusion) | 1.9 cm/s | (Scona et al., 2018, Table I) |
| CF (Co-Fusion) | 2.3 cm/s | (Scona et al., 2018, Table I) |
| BaMVO (Kim et al.) | 無數值未報告註記(擷取紀錄):not reported (BaMVO shown only for sequences evaluated in its original publication) | (Scona et al., 2018, Table I) |
| SF (StaticFusion)本方法原文提出 | 2.3 cm/s | (Scona et al., 2018, Table I) |
Scona et al., 2018 · Table II 本方法 11 筆
指標Trans. ATE RMSE (cm)
表格設定(擷取紀錄原文):TUM (Freiburg) RGB-D sequences grouped as static (fr1), low dynamic (fr3/sit) and high dynamic (fr3/walk) environments; StaticFusion and VO-SF at QVGA, ElasticFusion and Co-Fusion at their default VGA; fr3/walk_halfsphere* skips the first 5 s of high dynamics (Scona et al., 2018, Table II)
Trans. ATE RMSE (cm),TUM RGB-D (Freiburg) · fr1/xyz
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Scona et al., 2018 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Scona et al., 2018, Table II)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| VO-SF (Jaimez et al. joint visual odometry and scene flow) | 5.1 cm | (Scona et al., 2018, Table II) |
| EF (ElasticFusion) | 1.2 cm | (Scona et al., 2018, Table II) |
| CF (Co-Fusion) | 1.4 cm | (Scona et al., 2018, Table II) |
| SF (StaticFusion)本方法原文提出 | 1.4 cm | (Scona et al., 2018, Table II) |
Palazzolo et al., 2019 · Table II 本方法 6 筆
指標Absolute Trajectory Error (RMS) [m]
表格設定(擷取紀錄原文):TUM RGB-D dynamic scenes; ATE RMS; ReFusion uses virtual depth from 10 frames (about 0.3 s delay) to fill invalid depth; ReFusion, SF and MF are dense, DynaSLAM is feature-based; StaticFusion lost track on walking halfsphere (Palazzolo et al., 2019, Table II)
Absolute Trajectory Error (RMS) [m],TUM RGB-D · sitting static
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Palazzolo et al., 2019 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Palazzolo et al., 2019, Table II)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Ours (ReFusion)原文提出 | 0.009 m | (Palazzolo et al., 2019, Table II) |
| SF (StaticFusion)本方法 | 0.014 m | (Palazzolo et al., 2019, Table II) |
| DS (G) (DynaSLAM geometric) | 0.009 m | (Palazzolo et al., 2019, Table II) |
| DS (N+G) (DynaSLAM neural network + geometric) | 0.007 m | (Palazzolo et al., 2019, Table II) |
| MF (MaskFusion, values from its paper) | 0.021 m | (Palazzolo et al., 2019, Table II) |
其他比較組
來源
Scona et al., 2018
(2018)StaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3849-3856
DOI 10.1109/icra.2018.8460681程式碼
同儕審查已出版已讀全文近十年查證後修正
相關版本
- 程式碼釋出:raluca-scona/StaticFusion (built on the ElasticFusion code base) https://github.com/raluca-scona/StaticFusion
程式碼:https://github.com/raluca-scona/StaticFusion(授權:GPL-3.0 (GPL-LICENSE.txt), with reused ElasticFusion parts under the ElasticFusion licence (LICENSE.txt checked))。有公開程式碼不等於已被重現,也不代表目前版本與論文版本相同。