SurfelMeshing
SurfelMeshing 假設相機已校正且位姿由外部 SLAM 提供，不把深度融合進體素體積，而是融合成稠密面元（surfel）雲，再在背景非同步地對平滑後的面元做局部三角化，產生頂點即為面元的網格。作者在 ElasticFusion 式的面元重建上加入兩個去雜訊步驟：沿法向與鄰近面元的正則化，以及在觀測邊界漸進混合深度差以避免表面斷裂；並提出只在失效三角形附近重新三角化的增量演算法，使網格能隨迴圈閉合造成的面元變形快速更新。由於面元依輸入影像解析度建立，網格與色彩解析度會隨觀測距離調整，也能重建體素法難以保留的薄物體。
本頁內容
Online mesh reconstruction from a dense surfel cloud: ElasticFusion-style surfel fusion with new normal-direction regularization and observation-boundary blending, plus asynchronous local (re)triangulation so the mesh follows loop-closure deformations, adapts its resolution to the input and keeps thin objects; poses come from an external SLAM system.
技術屬性
欄位內容為文獻擷取紀錄的原文用語（英文），以原文為據；「未查證」表示本研究尚未讀到該資訊，不代表該方法不具備此能力。
| 感測輸入 | RGB-D camera (mainly Microsoft Kinect v1 sequences of the TUM RGB-D benchmark; pre-registered CoRBS sequences; synthetic ICL-NUIM depth with simulated noise) |
|---|---|
| 原文測試平台 | real RGB-D sequences from TUM RGB-D, CoRBS and ETH3D (capture platform not described in the paper; ETH3D poses from BAD SLAM)、simulation (ICL-NUIM synthetic living room) |
| 狀態估計 | No pose estimation of its own: calibrated camera and poses from an external SLAM system (ElasticFusion in the implementation; BAD SLAM poses for the ETH3D examples) |
| 資料關聯 | Projective association of each surfel with the pixel it projects to and the nearest neighbouring pixel; surfels classified as conflicting, occluded or supported using a depth uncertainty interval of plus or minus 5% of the measured depth and normal checks |
| 時間表示 | discrete camera poses |
| 去畸變 | 不適用 (RGB-D input) |
| 迴圈閉合 | Adopts ElasticFusion's loop-closure handling: the surfel cloud is deformed based on surfel timestamps, offsets are averaged among neighbours for 100 iterations, and affected mesh regions are remeshed; the public code excludes loop closure |
| 全域最佳化 | none of its own; relies on the SLAM system's non-rigid surfel deformation |
| 地圖表示 | dense surfel cloud at input image resolution (position, normal, colour, confidence, radius, creation and update timestamps, denoised position, four neighbours) indexed by a lazily updated compressed octree; a triangle mesh whose vertices are the surfels |
| 先驗資訊 | none |
| 可輸出幾何 | coloured triangle mesh with vertices at the surfels, updated online; not guaranteed manifold |
| 計算需求 | GPU (CUDA 8.0) for surfel reconstruction and denoising at frame rate, CPU threads for asynchronous meshing and remeshing; tested on Intel Core i7 6700K with GeForce GTX 1080; average remeshing iteration 212 ms; a loop closure over 3.1 million surfels takes about 680 ms (Sec. 4, 5.4) |
使用設備
原文使用的感測器、運算硬體與載具（equipment）。型號保留原文寫法，連結到設備頁中同一型號的歸併名稱；角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測（reference or ground truth）與比較對象設備。
| 類別 | 型號（原文寫法） | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| RGB-D 相機 | Kinect v1 | 資料集感測器 | TUM RGB-D | 640x480 images; depth beyond 3 m dropped in preprocessing | (Schöps et al., 2020, Sec. 4; Sec. 5.3) |
| RGB-D 相機 | Kinect v2歸入：Microsoft Kinect v2 | 資料集感測器 | CoRBS | pre-registered CoRBS sequences (sensor named in the title of reference [46]) | (Schöps et al., 2020, Sec. 5; ref. [46]) |
| 運算硬體 | PC with Intel Core i7 6700K and MSI Geforce GTX 1080 Gaming X 8G | 執行運算平台 | 未標示 | surfel reconstruction and denoising in CUDA 8.0 on the GPU; meshing on CPU | (Schöps et al., 2020, Sec. 4; Sec. 5) |
作者報告的優勢與限制
優勢
- With ground-truth trajectories on ICL-NUIM (1 cm threshold) accuracy is higher than InfiniTAM and FastFusion on most sequences; with loop closures completeness is higher than ElasticFusion on kt0 to kt2 (Table 2)
- Lowest mean curvature, i.e. smoothest surfaces, of all compared methods (Tables 1 and 2)
- Incremental remeshing yields mesh quality close to meshing from scratch at far lower cost; meshing the final cloud from scratch takes about 5.6 s (Table 1; Fig. 10)
- Reconstructs thin objects and colours at the camera's resolution (Sec. 5.3; Figs. 14 and 15)
限制
- Meshes are not guaranteed to be manifold and can contain small holes where smoothing is insufficient (Sec. 6; Fig. 18)
- Large loop-closure deformations adopted from ElasticFusion may rip surfaces apart (Sec. 6)
- Less complete than the TSDF methods when ground-truth poses are used; ElasticFusion is partly more accurate (Table 2; Sec. 5.2)
- Loop closures cause short disruptions, about 680 ms for 3.1 million surfels; cost of the external SLAM is not included in timings (Sec. 5.4)
- Depth beyond 3 m is dropped in preprocessing for the Kinect v1 data (Sec. 4)
營建工程相關證據
論文未涉及營建場景，評估以 TUM RGB-D、ICL-NUIM 與 CoRBS 室內小場景為主。它能在 SLAM 過程中持續輸出可隨迴圈閉合更新的網格，並保留薄物體與相機解析度的色彩，可作為室內構件即時網格預覽的參考；但前處理捨棄 3 m 以外的深度，大型室內空間或工地的適用性未經驗證（推論）。
原文驗證環境：公開基準、模擬
報告的性能數據
以下是原文作者報告的性能數值（author-reported results），不是本研究重新量測的結果。每張圖只並列同一個比較組（comparison group，同一張表、同一組實驗設定）內的方法；不同比較組之間的數值不可直接比較，也不構成排名。
本方法共出現在 5 個比較組，合計 30 筆紀錄。以下列出本方法紀錄最多的 4 組，其餘 1 組列在最後，並連到性能比較頁。
Schöps et al., 2020 · Table 2 (ground-truth trajectories) 本方法 12 筆
表格設定（擷取紀錄原文）：ICL-NUIM living room with simulated depth noise; ground-truth trajectories used and loop-closure handling disabled for all methods; reconstructions aligned to the ground-truth model with point-to-plane ICP; evaluation threshold 1 cm; 'smoothed' = same bilateral filter as SurfelMeshing preprocessing (Schöps et al., 2020, Table 2 (ground-truth trajectories))
Accuracy [%] (share of reconstructed surfels within 1 cm of ground truth)，ICL-NUIM · kt0
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Schöps et al., 2020 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Schöps et al., 2020, Table 2 (ground-truth trajectories))
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| InfiniTAM [29] | 76.4% | (Schöps et al., 2020, Table 2) |
| InfiniTAM [29] - smoothed | 78.3% | (Schöps et al., 2020, Table 2) |
| FastFusion [27] | 85.5% | (Schöps et al., 2020, Table 2) |
| FastFusion [27] - smoothed | 75.9% | (Schöps et al., 2020, Table 2) |
| ElasticFusion [17] | 96.2% | (Schöps et al., 2020, Table 2) |
| ElasticFusion [17] - smoothed | 95.7% | (Schöps et al., 2020, Table 2) |
| SurfelMeshing (Ours)本方法原文提出 | 93.5% | (Schöps et al., 2020, Table 2) |
Schöps et al., 2020 · Table 2 (loop-closure trajectories) 本方法 9 筆
表格設定（擷取紀錄原文）：ICL-NUIM living room with simulated depth noise; trajectories estimated with ElasticFusion including loop closures (kt3 omitted because ElasticFusion failed); aligned with point-to-plane ICP; threshold 1 cm; InfiniTAM and FastFusion cannot handle loop closures and have no values (Schöps et al., 2020, Table 2 (loop-closure trajectories))
Accuracy [%] (share of reconstructed surfels within 1 cm of ground truth)，ICL-NUIM · kt0
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Schöps et al., 2020 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Schöps et al., 2020, Table 2 (loop-closure trajectories))
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| ElasticFusion [17] | 95.8% | (Schöps et al., 2020, Table 2) |
| ElasticFusion [17] - smoothed | 96.8% | (Schöps et al., 2020, Table 2) |
| SurfelMeshing (Ours)本方法原文提出 | 87.2% | (Schöps et al., 2020, Table 2) |
Schöps et al., 2020 · Table 1 本方法 6 筆
表格設定（擷取紀錄原文）：Mesh quality on TUM RGB-D reconstructions (Kinect v1 data; the trajectories used for Table 1 are not specified); truncated extract of Table 1: full method and the two volumetric baselines, boundary-vertex share and mean curvature only (Schöps et al., 2020, Table 1)
bdry: amount of vertices on a mesh boundary，TUM RGB-D · fr1/desk
只並列這張表在相同設定下報告的方法；以「本方法：」開頭者為本頁方法。失敗、未執行與未報告以標記呈現，不是 0。
按 Tab 進入圖表後，用上下方向鍵逐一瀏覽各類別，Esc 關閉提示框；也可開啟表格檢視閱讀全部數值。
這些是 Schöps et al., 2020 在此表設定下報告的數值（author-reported results），只能在同一個比較組內對照，不代表方法在其他資料或設定下的表現。
資料來源作者報告值(Schöps et al., 2020, Table 1)
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| Ours (regularization, blending and remeshing enabled)本方法原文提出 | 2.3% | (Schöps et al., 2020, Table 1) |
| InfiniTAM [29] | 9.7% | (Schöps et al., 2020, Table 1) |
| FastFusion [27] | 17.7% | (Schöps et al., 2020, Table 1) |
Schöps et al., 2020 · Text Sec.5.4 本方法 2 筆
資料集與序列TUM RGB-D · fr3/long_office_household
表格設定（擷取紀錄原文）：Performance on the TUM fr3/long office household sequence at 640x480; remeshing statistics averaged over iterations (Schöps et al., 2020, Text Sec.5.4)
average time per remeshing iteration，TUM RGB-D · fr3/long_office_household
這張表在此指標與資料序列只列出本方法一筆，沒有可並列的其他方法，因此不畫圖，數值與出處見下表。這是 Schöps et al., 2020 在此表設定下報告的數值（author-reported results），不代表方法在其他資料或設定下的表現。
| 方法（原文寫法） | 報告值 | 出處 |
|---|---|---|
| SurfelMeshing (Ours)本方法原文提出硬體：Intel Core i7 6700K + GeForce GTX 1080 (external SLAM cost excluded) | 212 ms | (Schöps et al., 2020, Sec. 5.4; Fig. 19) |
其他比較組
列出其餘 1 個比較組
來源
Schöps et al., 2020
（2020）SurfelMeshing: Online Surfel-Based Mesh ReconstructionIEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10):2494-2507
DOI 10.1109/tpami.2019.2947048arXiv 1810.00729程式碼
同儕審查已出版已讀全文近十年查證後修正
相關版本
- accepted manuscript：SurfelMeshing (arXiv v2, author's accepted version with IEEE notice) https://arxiv.org/abs/1810.00729
- 程式碼釋出：puzzlepaint/surfelmeshing (loop closure functionality excluded) https://github.com/puzzlepaint/surfelmeshing
程式碼：https://github.com/puzzlepaint/surfelmeshing（授權：BSD-3-Clause style licence (LICENSE file checked; copyright ETH Zurich, Thomas Schoeps)）。有公開程式碼不等於已被重現，也不代表目前版本與論文版本相同。