NeRF-SLAM
NeRF-SLAM 把稠密單眼 SLAM 與即時雜湊式神經輻射場串接：追蹤端直接採用 DROID-SLAM 的學習式光流與稠密光束法平差，並依 σ-Fusion 的做法由 Hessian 結構計算每個深度與位姿的邊際共變異數。建圖端以 Instant-NGP 表示場景，損失同時包含顏色誤差與以共變異數加權的深度誤差，使雜訊大的單眼深度不致把幾何拉偏，並在同一執行緒中微調關鍵影格位姿。兩個執行緒平行運作，在單張 RTX 2080 Ti 上約每秒 10 影格，但系統沒有迴圈閉合。
本頁內容
Real-time monocular pipeline that feeds DROID-SLAM poses, dense depths and their marginal covariances into an Instant-NGP radiance field trained with a covariance-weighted depth loss, jointly refining poses and map; no loop closure.
技術屬性
欄位內容為文獻擷取紀錄的原文用語（英文），以原文為據；「未查證」表示本研究尚未讀到該資訊，不代表該方法不具備此能力。
| 感測輸入 | monocular camera |
|---|---|
| 原文測試平台 | simulation (Replica rendered sequences and Blender Cube-Diorama) |
| 狀態估計 | DROID-SLAM dense bundle adjustment over a sliding window of at most 8 keyframes (Schur complement and Cholesky solve), with marginal covariances of dense depths and poses computed as in sigma-Fusion; the mapping thread minimizes photometric plus covariance-weighted depth loss jointly over poses and radiance-field parameters (Sec. III-A to III-C) |
| 資料關聯 | dense learned optical flow with per-measurement weights from a RAFT-style ConvGRU (DROID-SLAM) (Sec. III-A) |
| 時間表示 | discrete poses (keyframes) |
| 去畸變 | 不適用 |
| 迴圈閉合 | none described; tracking uses only a sliding window of keyframes (Sec. III-C) |
| 全域最佳化 | none in tracking; the mapping thread optimizes all received keyframes (poses and map) through the rendering losses (Sec. III-B to III-C) |
| 地圖表示 | Instant-NGP hash-based hierarchical volumetric neural radiance field (density and colour), supervised by RGB and depth weighted by its marginal covariance (Sec. III-B, Sec. III-D) |
| 先驗資訊 | pretrained DROID-SLAM weights for tracking (Sec. III-D) |
| 可輸出幾何 | keyframe poses, dense keyframe depth maps with uncertainty and a radiance field rendered to colour and depth; no mesh or point-cloud accuracy evaluation, geometry assessed by rendered Depth L1 (Sec. IV-C) |
| 計算需求 | single NVIDIA RTX 2080 Ti (11 GB) shared by tracking and mapping; about 10 FPS overall at 640x480 (tracking 15 FPS, mapping 10 FPS; the section also states 12 FPS); needs about 11 GB GPU memory (Sec. III-D, Sec. IV-E, Sec. V) |
使用設備
原文使用的感測器、運算硬體與載具（equipment）。型號保留原文寫法，連結到設備頁中同一型號的歸併名稱；角色依原文用途分為方法輸入、資料集感測器、執行運算平台、參考或真值量測（reference or ground truth）與比較對象設備。
| 類別 | 型號（原文寫法） | 角色 | 資料集 | 原文規格 | 出處 |
|---|---|---|---|---|---|
| 運算硬體 | RTX 2080 Ti | 執行運算平台 | 未標示 | GPU with 11 Gb memory, used for tracking and mapping | (Rosinol et al., 2023, Sec. III-D) |
論文圖片
只收錄原文以開放授權（open license）釋出的圖片，並依授權條件標示出處、圖號、授權與修改方式。
Fig. 2系統架構：DROID-SLAM 追蹤迴路、邊際共變異數計算與以 Instant-NGP 擬合神經輻射場的資訊流
出處：Rosinol et al., 2023，Fig. 2。授權：CC BY 4.0 (arXiv v1 version of the figure; the IROS version is copyright IEEE)。原始圖檔。修改：縮小至寬度不超過 1400 px，並轉存為 WebP 格式。
作者報告的優勢與限制
優勢
- Replica average Depth L1 9.29 cm and PSNR 42.03 dB from monocular input, versus 14.18 cm and 17.76 dB for NICE-SLAM without depth (IROS version Table I; arXiv v1 printed 4.49 cm and 41.40 dB)
- Better geometric accuracy than TSDF-Fusion (21.88 cm) and sigma-Fusion (20.10 cm) built from the same poses and depths (Table I)
- Up to 178% better PSNR (office-1) and 75% better Depth L1 (room-2) than NICE-SLAM (abstract, Sec. IV-C)
- Uncertainty weighting avoids the slower and biased convergence seen when raw dense depths supervise the field (Sec. IV-D, Figs. 4-5)
- Runs at about 10 FPS with tracking and mapping in parallel on one GPU (Sec. IV-E)
限制
- Needs about 11 GB of GPU memory because of dense correlation volumes and hierarchical grids, which can be prohibitive for low-compute robots such as drones (Sec. V)
- NICE-SLAM supervised with ground-truth depth has a much lower average Depth L1 (4.08 cm) than NeRF-SLAM (9.29 cm), and in office-1 NeRF-SLAM (16.32 cm) is worse than NICE-SLAM without depth (10.24 cm) (Table I)
- No trajectory accuracy (ATE) is reported and evaluation uses only synthetic Replica and Cube-Diorama data (Sec. IV) (inference)
- No loop closure or global bundle adjustment; GO-SLAM later points out this limitation (Sec. III-C; GO-SLAM Sec. 2) (inference)
營建工程相關證據
論文未涉及營建場域；評估只使用 Replica 渲染序列與 Blender 合成的 Cube-Diorama，幾何品質以渲染深度的 L1 誤差代替，未報告軌跡誤差，也沒有網格或點雲對獨立參考量測的精度。其以單眼相機估計深度並依共變異數加權的做法，對低成本影像記錄工地有參考價值（推論），但約 11 GB GPU 記憶體需求與缺乏迴圈閉合限制了在大範圍工地的使用。
原文驗證環境：公開基準、模擬
報告的性能數據
性能數據仍在分批查證，目前尚未收錄此方法的報告值。
來源
Rosinol et al., 2023
（2023）NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3437-3444
DOI 10.1109/iros55552.2023.10341922arXiv 2210.13641程式碼
同儕審查已出版已讀全文近十年查證後修正
相關版本
- version of record：IROS 2023 proceedings (IEEE Xplore 10341922) https://doi.org/10.1109/IROS55552.2023.10341922
- 預印本：arXiv:2210.13641v1 https://arxiv.org/abs/2210.13641
- 程式碼釋出：ToniRV/NeRF-SLAM https://github.com/ToniRV/NeRF-SLAM
程式碼：https://github.com/ToniRV/NeRF-SLAM（授權：BSD-2-Clause (GitHub license detection of LICENSE.BSD)）。有公開程式碼不等於已被重現，也不代表目前版本與論文版本相同。