BundleFusion
BundleFusion 在每一影格都考慮完整的 RGB-D 歷史資料，以分塊（chunk）的階層式區域到全域最佳化，結合稀疏 SIFT 特徵與稠密幾何、光度對應，即時求得經 BA 的全域位姿。位姿更新後，系統即時將受影響影格從 TSDF 中移除並以新位姿重新融合（re-integration），使稠密模型保持全域一致。由於每張影格都與全部歷史比對，迴圈閉合是隱式處理的。
本頁內容
BundleFusion solves real-time globally bundle-adjusted poses over the whole RGB-D history with a hierarchical sparse-plus-dense optimisation and de-/re-integrates frames in a hashed TSDF whenever poses change.
技術屬性
欄位內容為文獻擷取紀錄的原文用語（英文），以原文為據；「未查證」表示本研究尚未讀到該資訊，不代表該方法不具備此能力。
| 感測輸入 | RGB-D |
|---|---|
| 原文測試平台 | hand-held Structure Sensor mounted to an iPad Air, streaming compressed RGB-D over a wireless network to a desktop that runs the optimisation and streams visual feedback back to the iPad、public datasets: TUM RGB-D (hand-held Kinect), SUN3D (Asus Xtion), NYU2 (Kinect), synthetic ICL-NUIM and augmented ICL-NUIM |
| 狀態估計 | Two-level hierarchical pose-only optimisation: intra-chunk alignment of 11 consecutive frames (1-frame overlap) and inter-chunk alignment of chunk keyframes with aggregated feature sets; energy = sparse SIFT correspondence distances plus dense photometric (luminance gradient) and point-to-plane geometric terms on 80x60 downsampled frames with linearly increasing dense weight; Gauss-Newton with a GPU data-parallel PCG solver (Jacobi preconditioner), warm-started from the previous frame; about 20 times faster than Ceres on a 101-keyframe sparse problem |
| 資料關聯 | GPU SIFT matched against all previous frames (about 150 features per frame, 250 per keyframe) and filtered by a Kabsch-based key point correspondence filter (max residual 0.02 m, condition number limit 100), a surface-area filter (0.032 m2) and two-sided dense geometric and photometric verification (Nmin = 5); after each optimisation all correspondences of a frame pair with residual above 0.05 m are pruned; dense terms use frame pairs within 60 degrees and nonzero overlap |
| 時間表示 | discrete poses (all frames, grouped in chunks) |
| 去畸變 | 原文未報告 |
| 迴圈閉合 | implicit: each frame is globally correlated to all previous frames, so no explicit loop detection; relocalization after gross tracking failure (abstract; introduction) |
| 全域最佳化 | real-time global pose optimisation with on-the-fly TSDF de-integration and re-integration when poses change; 3D keypoint positions stay fixed (only camera poses are optimised), and the dense term at the global keyframe level is run only after the user ends scanning, so online global alignment relies on sparse correspondences with dense terms inside chunks |
| 地圖表示 | TSDF in sparse voxel hashing with 8x8x8 voxel blocks, default 4 mm voxels (1 cm also evaluated); every frame stored with integrated and optimised poses and the 10 frames with the largest pose change re-integrated per new frame |
| 先驗資訊 | none |
| 可輸出幾何 | textured surface mesh from the TSDF (authors contrast it with the point-cloud output of ElasticFusion) plus globally optimised per-frame poses |
| 計算需求 | Intel Core i7 3.4 GHz CPU with 32 GB RAM; NVIDIA GeForce GTX Titan X for volumetric reconstruction and GTX Titan Black for correspondence search and pose optimisation; CUDA 7.0; frame rate well beyond 30 Hz; CPU RAM grows linearly with sequence length (20 GB for Apt 0, 34.7 GB for a 14,785-frame SUN3D scan) while GPU memory is 1.9 GB (1 cm) or 5.3 GB (4 mm) for Apt 0 |
使用設備
原文使用的感測器、運算硬體與載具（equipment）。型號保留原文寫法，連結到設備頁中同一型號的歸併名稱；角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測（reference or ground truth）與比較對象設備。
| 類別 | 型號（原文寫法） | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| RGB-D 相機 | Structure Sensor | 方法輸入 | BundleFusion captured sequences (Apt 0 to 2, Copyroom, Office 0 to 3) | RGB-D stream at 30 Hz with 640x480 colour and depth; another colour resolution of 1296x968 mentioned for SIFT timing; footnote links structure.io and the acknowledgements thank Occipital for hardware donations, but the manufacturer is not named | (Dai et al., 2017a, Sec. 6; Sec. 7.3; Acknowledgments) |
| RGB-D 相機 | Kinect | 資料集感測器 | TUM RGB-D | hand-held Kinect sequences | (Dai et al., 2017a, Sec. 6 Quantitative Comparison) |
| RGB-D 相機 | Asus Xtion sensor | 資料集感測器 | SUN3D | 原文未報告 | (Dai et al., 2017a, Sec. 7.1) |
| RGB-D 相機 | Kinect | 資料集感測器 | NYU2 | all 464 scenes reconstructed qualitatively | (Dai et al., 2017a, Sec. 7.1) |
| 載具平台 | iPad Air | 方法輸入 | BundleFusion captured sequences | Structure Sensor mounted on it; receives live visual feedback; data sent over wireless network with zlib depth and jpeg colour compression | (Dai et al., 2017a, Sec. 6) |
| 運算硬體 | Intel Core i7 3.4GHz CPU (32GB RAM) | 執行運算平台 | 未標示 | desktop host | (Dai et al., 2017a, Sec. 6 Performance and Convergence) |
| 運算硬體 | NVIDIA GeForce GTX Titan X | 執行運算平台 | 未標示 | 12 GB; volumetric reconstruction | (Dai et al., 2017a, Sec. 6; Fig. 4) |
| 運算硬體 | GTX Titan Black | 執行運算平台 | 未標示 | correspondence search and global pose optimisation | (Dai et al., 2017a, Sec. 6; Fig. 4) |
| 其他 | calibrated motion capture system | 參考或真值量測 | TUM RGB-D | ground-truth camera poses of TUM RGB-D | (Dai et al., 2017a, Sec. 6 Quantitative Comparison) |
作者報告的優勢與限制
優勢
- Global consistency with quality described as on par with offline methods (abstract)
- Robust tracking with relocalization from gross failures (abstract; Fig. 6)
- Lowest ATE RMSE on all four ICL-NUIM living-room sequences (kt1 tied with VoxelHashing at 0.4 cm) and on three of four augmented ICL-NUIM sequences (Tables 3 and 7)
- Lowest surface error on ICL-NUIM, 0.5 to 0.8 cm mean distance (Table 6)
- Loop-closure precision reaches 100% after optimisation pruning at 39 to 48% recall on augmented ICL-NUIM (Table 2)
限制
- Sparse keypoint mismatches of a few pixels and depth noise can propagate into local misalignments (limitations)
- The current implementation runs on two GPUs (Titan X and Titan Black in the performance evaluation), and the current hardware configuration limits scans to about 25,000 RGB-D frames, roughly 14 minutes at 30 Hz (limitations; Fig. 4)
- On augmented ICL-NUIM Office 1 the method has difficulty closing the loop because part of the trajectory covers a wall with little to no colour features (ATE RMSE 15.3 cm) (Sec. 7.2; Table 7)
- Frames on untextured walls or with occluded sensor fail the correspondence filters and stay unregistered (Table 8)
- Parameters must be relaxed for noisier sensors such as Kinect (Sec. 6 Parameters)
營建工程相關證據
論文未報告營建測試；以作者自錄大型室內掃描與 TUM、ICL-NUIM 等基準評估。約 14 分鐘的資料量上限、雙 GPU 需求，以及主記憶體隨影格數線性成長（Apt 0 需 20 GB），限制其直接用於大範圍工地掃描（推論）。作者也指出缺乏色彩紋理的牆面會使迴圈難以閉合，並使部分影格無法通過對應篩選而未完成配準（Sec. 7.2、Table 8），與未粉刷牆面的室內工地情境相關（推論）。
原文驗證環境：公開基準、模擬、受控實驗、獨立參考量測
報告的性能數據
以下是原文作者報告的性能數值（author-reported results），不是本研究重新量測的結果。每張圖只並列同一個比較組（comparison group，同一張表、同一組實驗設定）內的方法；不同比較組之間的數值不可直接比較，也不構成排名。
本方法共出現在 15 個比較組，合計 85 筆紀錄。以下列出本方法紀錄最多的 4 組，其餘 11 組列在最後，並連到性能比較頁。
Dai et al., 2017a · Table 3 本方法 12 筆
指標ATE RMSE
表格設定（擷取紀錄原文）：ICL-NUIM living-room trajectories kt0 to kt3 with synthetic noise; ATE RMSE; comparator values match those printed in ElasticFusion Table II; Ours (s) sparse-only and Ours (sd) sparse plus local dense are ablations; Redwood runs offline without colour (Dai et al., 2017a, Table 3)
ATE RMSE，ICL-NUIM · kt0
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Dai et al., 2017a 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Dai et al., 2017a, Table 3)
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| DVO SLAM | 10.4 cm | (Dai et al., 2017a, Table 3) |
| RGB-D SLAM | 2.6 cm | (Dai et al., 2017a, Table 3) |
| MRSMap | 20.4 cm | (Dai et al., 2017a, Table 3) |
| Kintinuous | 7.2 cm | (Dai et al., 2017a, Table 3) |
| VoxelHashing | 1.4 cm | (Dai et al., 2017a, Table 3) |
| Elastic Fusion | 0.9 cm | (Dai et al., 2017a, Table 3) |
| Redwood (rigid) | 25.6 cm | (Dai et al., 2017a, Table 3) |
| BundleFusion ablation: Ours (s), sparse only本方法 | 0.9 cm | (Dai et al., 2017a, Table 3) |
| BundleFusion ablation: Ours (sd), sparse and local dense本方法 | 0.8 cm | (Dai et al., 2017a, Table 3) |
| BundleFusion (Ours)本方法原文提出 | 0.6 cm | (Dai et al., 2017a, Table 3) |
Dai et al., 2017a · Table 4 本方法 12 筆
指標ATE RMSE
表格設定（擷取紀錄原文）：TUM RGB-D ATE RMSE; ground truth from a calibrated motion capture system for hand-held Kinect sequences; for Kinect data the dense reprojection threshold is 0.3 m and residuals above 0.16 m are pruned; Redwood offline and geometry-only (Dai et al., 2017a, Table 4)
ATE RMSE，TUM RGB-D · fr1/desk
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
- 未報告（沒有數值，不是 0）
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Dai et al., 2017a 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Dai et al., 2017a, Table 4)
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| DVO SLAM | 2.1 cm | (Dai et al., 2017a, Table 4) |
| RGB-D SLAM | 2.3 cm | (Dai et al., 2017a, Table 4) |
| MRSMap | 4.3 cm | (Dai et al., 2017a, Table 4) |
| Kintinuous | 3.7 cm | (Dai et al., 2017a, Table 4) |
| VoxelHashing | 2.3 cm | (Dai et al., 2017a, Table 4) |
| Elastic Fusion | 2 cm | (Dai et al., 2017a, Table 4) |
| LSD-SLAM | 無數值未報告註記（擷取紀錄）：no value in source ('-') | (Dai et al., 2017a, Table 4) |
| Submap BA | 2.2 cm | (Dai et al., 2017a, Table 4) |
| Redwood (rigid) | 2.7 cm | (Dai et al., 2017a, Table 4) |
| BundleFusion ablation: Ours (s), sparse only本方法 | 1.9 cm | (Dai et al., 2017a, Table 4) |
| BundleFusion ablation: Ours (sd), sparse and local dense本方法 | 1.7 cm | (Dai et al., 2017a, Table 4) |
| BundleFusion (Ours)本方法原文提出 | 1.6 cm | (Dai et al., 2017a, Table 4) |
Schöps et al., 2019 · Table 3 本方法 8 筆
表格設定（擷取紀錄原文）：Synthetic renders of dense TUM RGB-D reconstructions along the original trajectories; each value aggregates ATE RMSE over seven synthetic datasets per category (avg. or med.); rs uses Kinect v1 shutter times (about 30.5 ms depth, 26.1 ms colour); async renders colour midway between depth frames (Schöps et al., 2019, Table 3)
ATE RMSE [cm], average over seven synthetic datasets，synthetic TUM RGB-D renders (7 datasets per category) · clean
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Schöps et al., 2019 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Schöps et al., 2019, Table 3)
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| BundleFusion本方法 | 0.34 cm | (Schöps et al., 2019, Table 3) |
| DVO SLAM | 0.32 cm | (Schöps et al., 2019, Table 3) |
| ElasticFusion | 1.11 cm | (Schöps et al., 2019, Table 3) |
| ORB-SLAM2 | 0.47 cm | (Schöps et al., 2019, Table 3) |
| BAD SLAM (Ours)原文提出 | 0.15 cm | (Schöps et al., 2019, Table 3) |
Han & Fang, 2018 · Table I 本方法 8 筆
指標ATE rmse (cm)
表格設定（擷取紀錄原文）：Localization accuracy on TUM RGB-D as ATE RMSE (Sturm et al.) in cm; alignment not stated (Han & Fang, 2018, Table I)
ATE rmse (cm)，TUM RGB-D · fr1/desk
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Han & Fang, 2018 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Han & Fang, 2018, Table I)
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| RGBD SLAM [3] (Endres et al.) | 2.3 cm | (Han & Fang, 2018, Table I) |
| ElasticFusion | 2 cm | (Han & Fang, 2018, Table I) |
| BundleFusion (on-line)本方法 | 1.7 cm | (Han & Fang, 2018, Table I) |
| BundleFusion (off-line)本方法 | 1.6 cm | (Han & Fang, 2018, Table I) |
| FlashFusion原文提出 | 1.9 cm | (Han & Fang, 2018, Table I) |
其他比較組
列出其餘 11 個比較組
- Han & Fang, 2018 · Table II
- Dai et al., 2017a · Text Sec. 6 Memory
- Dai et al., 2017a · Table 6
- Dai et al., 2017a · Table 7
- Wang et al., 2019 · Table I
- Han & Fang, 2018 · Table III
- Huang et al., 2024c · Table 1
- Peng et al., 2024 · Supp. Table 11
- Schöps et al., 2019 · Table 2
- Liso et al., 2024 · Table 2
- Teed & Deng, 2021 · Fig. 4 table
來源
Dai et al., 2017a
（2017）BundleFusion: Real-Time Globally Consistent 3D Reconstruction Using On-the-Fly Surface ReintegrationACM Transactions on Graphics, 36(3):1-18 (Crossref pages; article number 原文未報告)
DOI 10.1145/3054739arXiv 1604.01093程式碼
同儕審查已出版已讀全文經典查證後修正
相關版本
- 預印本：BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration (arXiv v1-v3) https://arxiv.org/abs/1604.01093
- 程式碼釋出：BundleFusion https://github.com/niessner/BundleFusion
程式碼：https://github.com/niessner/BundleFusion（授權：CC BY-NC-SA 4.0 (LICENSE.txt)）。有公開程式碼不等於已被重現，也不代表目前版本與論文版本相同。