D3VO
D3VO 在直接稀疏里程計（DSO）中三個層次加入深度網路：自監督的 DepthNet 預測深度使新點一開始就有公制尺度，並形成虛擬立體項；網路同時預測光度不確定度，用來取代傳統的殘差權重；PoseNet 預測的相對位姿則在前端追蹤當作先驗因子，在後端光度平差中當作位姿能量項。網路只以立體影片自監督訓練，並預測仿射亮度參數以處理曝光變化。系統仍是無迴圈閉合的單眼里程計，輸出軌跡與稀疏點雲。
本頁內容
Monocular direct sparse odometry (DSO) boosted by a self-supervised network that supplies metric depth (with a virtual stereo term), per-pixel photometric uncertainty as residual weights, and relative-pose priors for tracking and bundle adjustment; no loop closure.
技術屬性
欄位內容為文獻擷取紀錄的原文用語（英文），以原文為據；「未查證」表示本研究尚未讀到該資訊，不代表該方法不具備此能力。
| 感測輸入 | monocular camera at run time (stereo videos only for self-supervised network training) |
|---|---|
| 原文測試平台 | vehicle (KITTI)、UAV (EuRoC MAV) |
| 狀態估計 | DSO-style windowed sparse photometric bundle adjustment (Gauss-Newton) with three learned inputs: DepthNet depth initializes points at metric scale and adds a virtual stereo term, learned photometric uncertainty sets residual weights, and PoseNet relative poses act as front-end tracking prior factors and as a pose energy term in the back end (Sec. 3.2, Supp. C) |
| 資料關聯 | direct photometric residuals on sparse points with an 8-pixel neighbourhood pattern and Huber norm, weighted by the learned uncertainty (Sec. 3.2) |
| 時間表示 | discrete poses (keyframes) |
| 去畸變 | 不適用 |
| 迴圈閉合 | none |
| 全域最佳化 | none; windowed photometric bundle adjustment with marginalization as in DSO (Sec. 3.2) |
| 地圖表示 | sparse point set hosted in keyframes (DSO) with depths initialized from the network (Sec. 3.2, Fig. 2) |
| 先驗資訊 | self-supervised DepthNet (ResNet-18 encoder, ImageNet initialization) and PoseNet trained on stereo videos of the same datasets (KITTI Eigen split; EuRoC sequences not used for testing), predicting depth, photometric uncertainty, relative pose and affine brightness parameters (Sec. 3.1, Sec. 4, Supp. A-B) |
| 可輸出幾何 | camera trajectory and sparse point cloud (Fig. 2) |
| 計算需求 | networks trained with PyTorch on a single Titan X Pascal GPU at 512x256; VO runtime not reported (Supp. A) |
使用設備
原文使用的感測器、運算硬體與載具（equipment）。型號保留原文寫法，連結到設備頁中同一型號的歸併名稱；角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測（reference or ground truth）與比較對象設備。
| 類別 | 型號（原文寫法） | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| 運算硬體 | Titan X Pascal | 執行運算平台 | 未標示 | single GPU used to train DepthNet and PoseNet; VO runtime not reported | (Yang et al., 2020a, Supp. A) |
作者報告的優勢與限制
優勢
- Mean t_rel 0.82% on the paper's KITTI test split, below stereo DSO (0.89) and stereo ORB-SLAM2 (0.91) (Table 4)
- EuRoC mean ATE 0.10 with a single camera, comparable to monocular VIO such as VI-DSO (0.11) (Table 6)
- Deep pose integration strongly improves the hardest EuRoC sequences (V1_03 from 0.63 to 0.13 when adding Dp to Dd) (Table 6)
- Self-supervised depth network with brightness parameters and uncertainty outperforms Monodepth2 on KITTI and EuRoC depth (Tables 1-2)
限制
- Networks are trained on sequences of the same datasets used for testing; cross-scene generalization of monocular depth remains a challenge (Sec. 4.1, Tables 2-3)
- Without SE(3) alignment, ATE on KITTI can be large (for example 26.9 m on sequence 01) because early pose errors propagate (Supp. E, Supp. Table 4)
- No loop closure or global optimization (Sec. 3.2) (inference from the method description)
- Runtime of the VO system is not reported (inference)
營建工程相關證據
論文未涉及營建場域；只在 KITTI 車載與 EuRoC 飛行器資料上評估，且深度網路以同資料集的立體影片訓練，換到工地場景是否仍能提供可靠公制尺度未經驗證（推論）。系統只輸出稀疏點與軌跡，沒有迴圈閉合，主要價值在於示範學習式深度、不確定度與位姿先驗如何整合進直接法里程計。
原文驗證環境：公開基準
報告的性能數據
性能數據仍在分批查證，目前尚未收錄此方法的報告值。
來源
Yang et al., 2020a
（2020）D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1278-1289
DOI 10.1109/cvpr42600.2020.00136arXiv 2003.01060
同儕審查已出版已讀全文近十年查證後修正
相關版本
- 預印本：arXiv:2003.01060v2 https://arxiv.org/abs/2003.01060
- supplementary：CVPR 2020 supplementary material (CVF open access) https://openaccess.thecvf.com/content_CVPR_2020/supplemental/Yang_D3VO_Deep_Depth_CVPR_2020_supplemental.pdf