【點雲論文速讀】RevealNet: Seeing Behind Objects in RGB-D Scans

• 2020 年 4 月 10 日
• 筆記

點雲PCL免費知識星球，點雲論文速讀。

標題：RevealNet: Seeing Behind Objects in RGB-D Scans

作者：Ji Hou Angela Dai Matthias Nießner

來源：cvpr2020.

星球ID：Lionheart|點雲配准

歡迎各位加入免費知識星球，獲取PDF論文，歡迎轉發朋友圈分享快樂。

●論文摘要

一種端到端的RGB-D場景點雲補全方法。在3D重建中，經常出現不能完全掃描獨立目標，導致場景中幾何資訊的缺失，這些丟失的資訊嚴重影響了許多應用，例如：一個機器人需要未知幾何資訊完成精確的目標抓取，因此我們引入了語義補全的例子：在一個不完整的RGB-D場景中，我們嘗試探測獨立的目標並且猜測他們完整的目標幾何形狀，這將為場景交互帶來新的可能性，例如虛擬現實與機器人代理，我們引入RevealNet網路來完整這項任務，一個以數據為驅動的方法來探測目標並預測他們完整的幾何形狀，以為著將掃描的場景分解成獨立的擁有語義含義的對象，RevealNet是一個端到端的3D神經網路，能夠融合顏色和幾何特徵資訊，我們的三維網路的全卷積性質使得語義實例完成的推理在大型室內環境下的三維掃描只需一個前向通道。我們表明，預先完成完整的對象幾何圖形可以同時改善3D檢測和實例分割性能。

●論文圖集

使用RecealNet網路補全場景實例，網路中使用到了影像的顏色和場景的幾何資訊。場景中目標被檢測出來，同時預測被檢測目標完整的幾何形狀。

RevealNet網路結構

方法實例效果展示

方法效果對比

●英文摘要

During 3D reconstruction, it is often the case that people cannot scan each individual object from all views, resulting in missing geometry in the captured scan. This missing geometry can be fundamentally limiting for many applications, e.g., a robot needs to know the unseen geometry to perform a precise grasp on an object. Thus, we introduce the task of semantic instance completion: from an incomplete RGB-D scan of a scene, we aim to detect the individual object instances and infer their complete object geometry. This will open up new possibilities for interactions with objects in a scene, for instance for virtual or robotic agents. We tackle this problem by introducing RevealNet, a new data-driven approach that jointly detects object instances and predicts their complete geometry. This enables a semantically meaningful decomposition of a scanned scene into individual, complete 3D objects, including hidden and unobserved object parts.

RevealNet is an end-to-end 3D neural network architecture that leverages joint color and geometry feature learning. The fully-convolutional nature of our 3D network enables efficient inference of semantic instance completion for 3D scans at scale of large indoor environments in a single forward pass. We show that pre dicting complete object geometry improves both 3D detection and instance segmentation performance. We evaluate on both real and synthetic scan benchmark data for the new task, where we outperform state-of-the-art approaches by over 15 in mAP@0.5 on ScanNet, and over 18 in mAP@0.5 on SUNCG