Kintinuous
Kintinuous 以 GPU 上的循環緩衝(cyclical buffer)讓 TSDF 融合體積隨相機移動,使稠密融合可延伸到無界空間,並結合稠密幾何與光度約束估計位姿。偵測到迴圈後,以 as-rigid-as-possible 空間變形校正已建立的稠密地圖。作者報告可在數百公尺範圍內即時產生全域一致的表面重建。
本頁內容
Kintinuous shifts a TSDF volume with the camera via a GPU cyclical buffer, combines dense geometric and photometric tracking, and corrects the dense map after loop closure with as-rigid-as-possible deformation.
技術屬性
欄位內容為文獻擷取紀錄的原文用語(英文),以原文為據;「未查證」表示本研究尚未讀到該資訊,不代表該方法不具備此能力。
| 感測輸入 | RGB-D |
|---|---|
| 原文測試平台 | handheld |
| 狀態估計 | Frame-to-model point-to-plane ICP against the raycast TSDF combined with frame-to-frame dense photometric RGB-D alignment in a weighted sum (w_rgbd = 0.1), three-level pyramids, GPU tree reduction and CPU Cholesky solve; loop constraints enter an iSAM pose graph, whose optimised poses and matched SURF points constrain an embedded-deformation optimisation (weights 1, 10, 100, 100) solved by Gauss-Newton with CHOLMOD |
| 資料關聯 | dense geometric and photometric; projective data association (conclusion) |
| 時間表示 | discrete poses |
| 去畸變 | 原文未報告 (rolling shutter not modelled; authors note that projective data association limits the camera motions the front-end can handle, which also limits motion blur and rolling-shutter effects, and that real-time correction would add computation; conclusion) |
| 迴圈閉合 | Frames enter the DBoW (SURF) database when a combined rotation and translation motion metric exceeds 0.3; a candidate needs at least 35 FLANN SURF matches, a RANSAC 3-point transform with a 2.0 px reprojection threshold and at least 25% inliers refined by Levenberg-Marquardt, and a final ICP between voxel-downsampled clouds accepted when the mean squared correspondence error is below 0.01; the accepted constraint is added to iSAM and the dense map is corrected by as-rigid-as-possible embedded deformation |
| 全域最佳化 | iSAM incremental pose-graph optimisation plus non-rigid embedded deformation of all cloud-slice vertices; deformation nodes are sampled along the pose graph with sequential k = 4 connectivity and vertices are associated by back-traversal so that unrelated map regions are not linked; runs online without a final batch step, optionally on a subsampled pose graph |
| 地圖表示 | GPU TSDF of 512^3 voxels (6 bytes each: truncated float16 distance, uint8 weight, RGB) addressed with modulo arithmetic as a cyclical buffer that shifts with the camera; surface leaving the volume is extracted by axis-aligned raycasts into voxel-grid-filtered cloud slices tied to the pose that caused the shift and incrementally triangulated with Greedy Projection Triangulation; revisited areas are not re-fused |
| 先驗資訊 | none |
| 可輸出幾何 | Large-scale dense coloured surface as cloud slices and a triangle mesh; the seven hand-held datasets span 30 to 318 m and about 0.9 to 6.2 million vertices |
| 計算需求 | Desktop PC (Ubuntu 12.04) with Intel Core i7-3960X at 3.30 GHz, 16 GB RAM and nVidia GeForce 680GTX with 2 GB; the 512^3-voxel TSDF uses 768 MB of GPU memory. The frontend averages 29.94 ms per frame on fr1/desk at the chosen shift threshold of 16 voxels, below the 30 Hz sensor period; loop-closure latency (recognition to corrected map) is 0.99 to 11.56 s with an every-frame pose graph and 0.64 to 2.78 s with a subsampled pose graph over the six datasets |
使用設備
原文使用的感測器、運算硬體與載具(equipment)。型號保留原文寫法,連結到設備頁中同一型號的歸併名稱;角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測(reference or ground truth)與比較對象設備。
| 類別 | 型號(原文寫法) | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| RGB-D 相機 | commodity RGB-D camera (model not stated) | 方法輸入 | authors' seven hand-held datasets | 640x480 frames at 30 Hz; auto exposure and auto white balance enabled in all real datasets | (Whelan et al., 2015b, Sec. 2.5.1, 5.1, 5.3.1) |
| 載具平台 | hand-held RGB-D camera | 方法輸入 | authors' seven hand-held datasets | seven datasets from 30 to 318 m (coffee room, corridor, garden, outdoors, two floors, indoor and outdoor, apartment) | (Whelan et al., 2015b, Sec. 5.2; Table 5) |
| 運算硬體 | Intel Core i7-3960X | 執行運算平台 | 未標示 | 3.30 GHz, 16 GB RAM, Ubuntu 12.04 desktop | (Whelan et al., 2015b, Sec. 5.3) |
| 運算硬體 | nVidia GeForce 680GTX | 執行運算平台 | 未標示 | 2 GB GPU memory; the 512^3 TSDF volume uses 768 MB | (Whelan et al., 2015b, Sec. 2.2, 5.3) |
| 其他 | motion capture system (TUM RGB-D ground truth, model not stated) | 參考或真值量測 | TUM RGB-D | synchronised ground-truth poses | (Whelan et al., 2015b, Sec. 5.1) |
作者報告的優勢與限制
優勢
- Globally consistent surface reconstructions over hundreds of metres with a commodity RGB-D sensor (abstract)
- Fused volumetric map gives higher quality than raw RGB-D point clouds (abstract)
- Deformation-corrected maps agree with 2-pass reconstructions to 1.2 to 2.8 mm residual on six datasets and 19.0 mm on Apartment (Table 5)
- No post-processing is needed, whereas DVO SLAM needed 7.41 to 5501.58 s of post-processing and failed on five of the seven hand-held datasets (Table 6)
- ATE RMSE on TUM RGB-D typically within 3 cm of DVO SLAM, RGB-D SLAM and MRS (Sec. 5.1.1, Table 2)
限制
- Projective data association limits the motions the odometry front-end can handle (conclusion)
- Revisited areas are not re-integrated into the volume, causing aliasing where multiple passes occur (conclusion)
- Higher ATE on fr1/desk2 and fr1/room, linked to high angular velocity, motion blur and rolling shutter (Sec. 5.1, Table 1)
- Surface self-intersection is possible after deformation, though rarely observed (Sec. 5.2)
- Fixed-step raycasting causes artefacts near object edges that lower synthetic depth accuracy (Sec. 5.2.3)
- Volume placement can leave distant surfaces such as a ceiling unreconstructed (Fig. 17 caption)
營建工程相關證據
論文未報告營建現場測試;資料為作者自錄室內外序列與 RGB-D 基準。重複經過區域不重新融合而產生重疊(aliasing)的限制,對需要多次行經的工地掃描有直接意義(推論)。
原文驗證環境:公開基準、受控實驗、模擬
報告的性能數據
以下是原文作者報告的性能數值(author-reported results),不是本研究重新量測的結果。每張圖只並列同一個比較組(comparison group,同一張表、同一組實驗設定)內的方法;不同比較組之間的數值不可直接比較,也不構成排名。
本方法共出現在 41 個比較組,合計 264 筆紀錄。以下列出本方法紀錄最多的 4 組,其餘 37 組列在最後,並連到性能比較頁。
Whelan et al., 2015b · Table 1 本方法 30 筆
表格設定(擷取紀錄原文):TUM RGB-D ATE statistics of Kintinuous (m), mean over ten runs of each dataset; mean angular velocity (deg/s) of each sequence given in the table (Whelan et al., 2015b, Table 1)
ATE RMSE (m),TUM RGB-D · fr1/desk
這張表在此指標與資料序列只列出本方法一筆,沒有可並列的其他方法,因此不畫圖,數值與出處見下表。這是 Whelan et al., 2015b 在此表設定下報告的數值(author-reported results),不代表方法在其他資料或設定下的表現。
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Kintinuous本方法原文提出 | 0.0407 m | (Whelan et al., 2015b, Table 1) |
Handa et al., 2014 · Table II 本方法 16 筆
表格設定(擷取紀錄原文):Surface reconstruction error, noise-free living room: CloudCompare cloud/mesh distance after manual coarse alignment and ICP fine alignment to the densely sampled model; ICP odometry except the kt0 (DVO) column; TSDF volume 4.5 m, truncation 0.045 m (Handa et al., 2014, Table II)
cloud/mesh distance: perpendicular distance from each reconstructed vertex to the closest ground-truth model triangle (CloudCompare),ICL-NUIM · kt0 (lr)
只並列這張表在相同設定下報告的方法;以「本方法:」開頭者為本頁方法。失敗、未執行與未報告以標記呈現,不是 0。
按 Tab 進入圖表後,用上下方向鍵逐一瀏覽各類別,Esc 關閉提示框;也可開啟表格檢視閱讀全部數值。
這些是 Handa et al., 2014 在此表設定下報告的數值(author-reported results),只能在同一個比較組內對照,不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Handa et al., 2014, Table II)
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Kintinuous pipeline with DVO odometry [13] | 0.0662 m | (Handa et al., 2014, Table II) |
| Kintinuous pipeline with ICP odometry (as in KinectFusion and Kintinuous [3], [4])本方法 | 0.0612 m | (Handa et al., 2014, Table II) |
Handa et al., 2014 · Table VII 本方法 16 筆
表格設定(擷取紀錄原文):Surface reconstruction error, living room with simulated noise, all using ICP odometry; same CloudCompare cloud/mesh procedure (Handa et al., 2014, Table VII)
cloud/mesh distance: perpendicular distance from each reconstructed vertex to the closest ground-truth model triangle (CloudCompare),ICL-NUIM · kt0 (lr)
這張表在此指標與資料序列只列出本方法一筆,沒有可並列的其他方法,因此不畫圖,數值與出處見下表。這是 Handa et al., 2014 在此表設定下報告的數值(author-reported results),不代表方法在其他資料或設定下的表現。
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Kintinuous pipeline with ICP odometry (as in KinectFusion and Kintinuous [3], [4])本方法 | 0.0114 m | (Handa et al., 2014, Table VII) |
Whelan et al., 2015b · Table 5 本方法 14 筆
表格設定(擷取紀錄原文):Seven hand-held datasets: RMS residual of point-to-plane ICP between the deformation-corrected map and a 2-pass map rebuilt from the optimised pose graph (mm); '2-pass fast' uses a subsampled pose graph; a consistency measure, not accuracy against an independent reference (Whelan et al., 2015b, Table 5)
2-pass residual registration error (mm),authors' hand-held datasets · Coffee (30.18 m, 909422 vertices)
這張表在此指標與資料序列只列出本方法一筆,沒有可並列的其他方法,因此不畫圖,數值與出處見下表。這是 Whelan et al., 2015b 在此表設定下報告的數值(author-reported results),不代表方法在其他資料或設定下的表現。
| 方法(原文寫法) | 報告值 | 出處 |
|---|---|---|
| Kintinuous (every-frame pose graph)本方法原文提出 | 1.2 mm | (Whelan et al., 2015b, Table 5) |
其他比較組
列出其餘 37 個比較組
- Whelan et al., 2015b · Table 2
- Choi et al., 2015 · Table 7
- Handa et al., 2014 · Table III
- Handa et al., 2014 · Table IV
- Handa et al., 2014 · Table V
- Handa et al., 2014 · Table VI
- Whelan et al., 2015b · Table 7
- Whelan et al., 2015b · Table 8
- Whelan et al., 2015b · Table 9
- Mur-Artal & Tardos, 2017 · Table III
- Liso et al., 2024 · Table 2
- Sandström et al., 2023 · Table 3
- Keetha et al., 2024 · Table 1
- Whelan et al., 2015b · Table 10
- Whelan et al., 2015b · Table 11
- Dai et al., 2017a · Table 3
- Dai et al., 2017a · Table 4
- Dai et al., 2017a · Table 6
- Dai et al., 2017a · Table 7
- Choi et al., 2015 · Supp. Table 3
- Choi et al., 2015 · Supp. Table 4
- Choi et al., 2015 · Table 4
- Whelan et al., 2015a · Table I
- Whelan et al., 2015a · Table II
- Whelan et al., 2015a · Table III
- Yan et al., 2024 · Table 2
- Whelan et al., 2015b · Table 3
- Whelan et al., 2015b · Table 4
- Matsuki et al., 2024 · Table 1
- Yan et al., 2017 · Table 2
- Yan et al., 2017 · Table 4
- Yan et al., 2017 · Table 5
- Schöps et al., 2019 · Table 2
- Wang et al., 2023a · Table 4
- Sucar et al., 2021 · Table 3
- Zhu et al., 2022a · Table 2
- Teed & Deng, 2021 · Fig. 4 table
來源
Whelan et al., 2015b
(2015)Real-time large-scale dense RGB-D SLAM with volumetric fusionThe International Journal of Robotics Research, 34(4-5):598-626
DOI 10.1177/0278364914551008程式碼
同儕審查已出版已讀全文經典查證後修正
相關版本
- 程式碼釋出:Kintinuous https://github.com/mp3guy/Kintinuous
- 會議版:Robust real-time visual odometry for dense RGB-D mapping (ICRA 2013, pp. 5724-5731) 10.1109/ICRA.2013.6631400
- 會議版:Deformation-based loop closure for large scale dense RGB-D SLAM (IROS 2013, pp. 548-555) 10.1109/IROS.2013.6696405
- 研討會工作坊版:Kintinuous: Spatially Extended KinectFusion (RSS RGB-D Workshop 2012; no DOI) http://thomaswhelan.ie/Whelan12rssw.pdf
程式碼:https://github.com/mp3guy/Kintinuous(授權:custom licence, non-commercial purposes only (LICENSE.txt))。有公開程式碼不等於已被重現,也不代表目前版本與論文版本相同。