CNN-SLAM
CNN-SLAM 以 LSD-SLAM 的直接法關鍵影格架構為基礎，只在建立關鍵影格時用卷積網路預測稠密深度，並依目前相機與訓練相機的焦距比例調整尺度，再以後續影格的小基線立體匹配依不確定度加權修正深度。低紋理區保留網路預測、高梯度區由立體量測主導，因此單眼 SLAM 可取得絕對尺度，在純旋轉運動下也能重建。關鍵影格以位姿圖最佳化，深度圖可再融合成含語意標籤的三維模型。
本頁內容
Monocular direct SLAM (LSD-SLAM style) whose key-frame depth is initialized by CNN prediction, rescaled by focal length and refined by uncertainty-weighted small-baseline stereo, recovering absolute scale, handling pure rotation and fusing semantic labels; key-frames are pose-graph optimized.
技術屬性
欄位內容為文獻擷取紀錄的原文用語（英文），以原文為據；「未查證」表示本研究尚未讀到該資訊，不代表該方法不具備此能力。
| 感測輸入 | monocular camera |
|---|---|
| 原文測試平台 | simulation (ICL-NUIM)、原文未報告 (TUM RGB-D sequences recorded with a Kinect; own office sequence setup not described) |
| 狀態估計 | LSD-SLAM-style direct key-frame tracking: weighted Gauss-Newton minimization of Huber-weighted photometric residuals on high-gradient pixels against the nearest key-frame; key-frame depth from a CNN (ResNet-50 fully convolutional network of Laina et al.) scaled by the focal-length ratio and refined by uncertainty-weighted fusion of small-baseline stereo depth; key-frame pose-graph optimization (Sec. 3.1-3.4) |
| 資料關聯 | direct photometric alignment; per-frame depth from 5-pixel epipolar matching for refinement (Sec. 3.1, Sec. 3.4) |
| 時間表示 | discrete poses (key-frames) |
| 去畸變 | 不適用 |
| 迴圈閉合 | pose-graph edges added between a new key-frame and existing key-frames with a similar field of view (small relative pose); no appearance-based place recognition described (Sec. 3.3) |
| 全域最佳化 | pose-graph optimization of key-frame poses at each new key-frame (Sec. 3.3) |
| 地圖表示 | dense per-key-frame depth and uncertainty maps fused into a global 3D model with optional semantic labels using the fusion scheme of its reference [27] (Sec. 3.5) |
| 先驗資訊 | CNN depth prediction and 4-class semantic segmentation trained on NYU Depth v2 (indoor, Kinect), ResNet-50 initialized on ImageNet; depth scaled by the ratio of current to training focal length (Sec. 3.2-3.3, Sec. 4) |
| 可輸出幾何 | camera trajectory, dense key-frame depth maps and a fused, optionally semantically labelled, 3D reconstruction (Sec. 3.5, Figs. 1 and 6) |
| 計算需求 | Intel Xeon 2.4 GHz CPU with 16 GB RAM and Nvidia Quadro K5200 (8 GB); CNN runs on the GPU and the other stages on two CPU threads; described as real-time but no frame rate is reported (Sec. 4) |
使用設備
原文使用的感測器、運算硬體與載具（equipment）。型號保留原文寫法，連結到設備頁中同一型號的歸併名稱；角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測（reference or ground truth）與比較對象設備。
| 類別 | 型號（原文寫法） | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| RGB-D 相機 | Kinect | 資料集感測器 | TUM RGB-D; NYU Depth v2 | TUM RGB-D is 'acquired with a Kinect sensor'; NYU Depth v2 ground truth from a Microsoft Kinect camera | (Tateno et al., 2017, Sec. 4) |
| 運算硬體 | Intel Xeon CPU at 2.4GHz | 執行運算平台 | 未標示 | desktop PC with 16GB of RAM | (Tateno et al., 2017, Sec. 4) |
| 運算硬體 | Nvidia Quadro K5200 | 執行運算平台 | 未標示 | 8GB VRAM; runs CNN depth prediction and semantic segmentation | (Tateno et al., 2017, Sec. 4) |
作者報告的優勢與限制
優勢
- Lowest average ATE (0.246 m) across 9 ICL-NUIM and TUM sequences, below LSD-SLAM bootstrapped with ground-truth depth (0.562 m) and ORB-SLAM (0.643 m) (Table 1)
- Highest average share of correct depth (22.464%) versus 3.032% for bootstrapped LSD-SLAM and 18.452% for raw CNN depth fusion (Table 1)
- Reconstructs scenes under mostly pure rotation (TUM fr1/rpy) where LSD-SLAM is noisy and ORB-SLAM fails to initialize (Sec. 4.2, Fig. 5)
- First joint 3D and semantic reconstruction from a monocular camera, per the authors (Sec. 4.3)
限制
- Bootstrapped LSD-SLAM has lower ATE on 4 of 9 sequences (for example TUM seq3: 0.037 versus 0.214 m) (Table 1)
- Only 12-37% of depth values fall within 10% of ground truth even after refinement (Table 1)
- Absolute scale depends on a CNN trained on NYU Depth v2 and on a focal-length correction; performance with other cameras and scene types is shown only on ICL-NUIM and TUM (Sec. 3.3, Sec. 4) (inference)
- Future work: closing the loop between geometric refinement and depth prediction (Sec. 5)
營建工程相關證據
論文未涉及營建場域；定量評估只用合成 ICL-NUIM 與 TUM 辦公室序列，平均 ATE 約 0.25 m，且深度誤差在 10% 內的比例僅約兩成，與施工量測所需的公分級精度差距很大。其以學習式深度先驗恢復單眼絕對尺度的想法，是後續單眼神經 SLAM 的重要源頭（推論）。
原文驗證環境：公開基準、模擬
報告的性能數據
性能數據仍在分批查證，目前尚未收錄此方法的報告值。
來源
Tateno et al., 2017
（2017）CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6565-6574
DOI 10.1109/cvpr.2017.695arXiv 1704.03489
同儕審查已出版已讀全文近十年
相關版本
- 預印本：arXiv:1704.03489v1 https://arxiv.org/abs/1704.03489