DeepFactors
DeepFactors 把 CodeSLAM 的學習式精簡深度編碼放進標準因子圖：每個關鍵影格的深度由 32 維編碼經以影像為條件的線性解碼器產生，位姿與編碼一起以 GTSAM 的 iSAM2 做批次最大後驗估計。關鍵影格間同時使用稠密光度誤差、BRISK 特徵重投影誤差與稀疏幾何深度一致性誤差三種因子，並有局部與以詞袋檢索的全域迴圈閉合。它在單張 GTX 1080 上可即時運作，但輸出深度解析度只有 256x192，且小編碼難以表示完全平坦的表面。
本頁內容
Real-time monocular dense SLAM that optimizes key-frame poses and learned 32-D depth codes in a GTSAM/iSAM2 factor graph with photometric, reprojection and geometric factors, plus local and bag-of-words global loop closure.
技術屬性
欄位內容為文獻擷取紀錄的原文用語（英文），以原文為據；「未查證」表示本研究尚未讀到該資訊，不代表該方法不具備此能力。
| 感測輸入 | monocular camera |
|---|---|
| 原文測試平台 | 原文未報告 (ScanNet, ICL-NUIM and TUM RGB-D sequences and a live camera; capture platforms not described) |
| 狀態估計 | batch MAP factor graph over key-frame poses and compact depth codes solved incrementally with iSAM2 in GTSAM; camera tracking by GPU direct whole-image SE(3) Lucas-Kanade against the closest key-frame; tracking and mapping interleaved; one-way frames refine the latest key-frame and are then marginalized (Sec. V) |
| 資料關聯 | three pairwise factor types: dense photometric error, BRISK keypoint reprojection error with Cauchy cost, and sparse geometric depth-consistency error with Huber cost (Sec. III) |
| 時間表示 | discrete poses (key-frames) |
| 去畸變 | 不適用 |
| 迴圈閉合 | local loops by a pose-based criterion within the last 10 key-frames; global loops by bag-of-words (DBoW2) candidates verified by tracking inliers and pose distance, closed with reprojection factors only (Sec. V-D) |
| 全域最佳化 | incremental batch optimization of all key-frames (iSAM2) with zero-code prior factors (Sec. V-B) |
| 地圖表示 | key-frame depth maps decoded linearly from a 32-dimensional learned code conditioned on the image (CodeSLAM-style variational auto-encoder) (Sec. III-IV, Sec. VI-A) |
| 先驗資訊 | network trained on about 1.4M ScanNet images with merged rendered and sensor depth, image size 256x192, code size 32; an explicit code-prediction path gives the initial depth of each key-frame (Sec. IV, Sec. VI-A) |
| 可輸出幾何 | key-frame poses and dense key-frame depth maps fused for visualization as reconstructions (Figs. 1 and 7) |
| 計算需求 | single NVIDIA GTX 1080 at 256x192; about 340 ms per new key-frame for network and Jacobian (16 ms forward pass), tracking at about 250 Hz; overall speed depends on connectivity, enabled factors and loop closures (Sec. VII) |
使用設備
原文使用的感測器、運算硬體與載具（equipment）。型號保留原文寫法，連結到設備頁中同一型號的歸併名稱；角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測（reference or ground truth）與比較對象設備。
| 類別 | 型號（原文寫法） | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| 運算硬體 | NVIDIA GTX 1080 | 執行運算平台 | 未標示 | single GPU running the network, CUDA kernels and visualization at 256x192 | (Czarnowski et al., 2020, Sec. VII) |
作者報告的優勢與限制
優勢
- Combining photometric, reprojection and geometric factors gives the best ATE and depth accuracy in the ScanNet ablation (Table I)
- Average 27.10% of depth pixels within 10% of truth on ICL-NUIM and TUM versus 19.77% for CNN-SLAM (Table II)
- Outperforms CodeSLAM on all and CNN-SLAM on four of five TUM fr1 sequences while running in real time (Table III)
- Standard factor-graph formulation allows other sensors to be added (Sec. VIII)
限制
- Small code size (32) cannot represent fully flat depth well, hurting the flat-wall TUM seq2 sequence (Sec. VI-C)
- Network Jacobian computation takes about 340 ms per key-frame; speed depends on enabled factors, and the geometric factor is disabled for fast exploration (Sec. VII)
- Monocular: trajectories and depth are scaled with the optimal scale from the TUM scripts for evaluation (Sec. VI-C)
- Low working resolution of 256x192 (Sec. VI-A, Sec. VII)
營建工程相關證據
論文未涉及營建場域；評估使用 ScanNet、ICL-NUIM 與 TUM RGB-D 室內資料，深度精度以落在真值 10% 以內的像素比例衡量，平均約 27%，且需以最佳尺度校正單眼結果。其價值在於示範學習式深度先驗可與因子圖、iSAM2 與迴圈閉合等傳統 SLAM 後端結合（推論），但解析度與精度都不足以直接產生施工量測用點雲。
原文驗證環境：公開基準、模擬
報告的性能數據
性能數據仍在分批查證，目前尚未收錄此方法的報告值。
來源
Czarnowski et al., 2020
（2020）DeepFactors: Real-Time Probabilistic Dense Monocular SLAMIEEE Robotics and Automation Letters, 5(2):721-728
DOI 10.1109/lra.2020.2965415arXiv 2001.05049程式碼
同儕審查已出版已讀全文近十年查證後修正
相關版本
- 預印本：arXiv:2001.05049v1 (accepted manuscript of the RA-L paper) https://arxiv.org/abs/2001.05049
- 程式碼釋出：jczarnowski/DeepFactors https://github.com/jczarnowski/DeepFactors
程式碼：https://github.com/jczarnowski/DeepFactors（授權：custom Imperial College licence for non-commercial, internal or academic research use (LICENSE file checked; GitHub reports NOASSERTION)）。有公開程式碼不等於已被重現，也不代表目前版本與論文版本相同。