【点云论文速读】RevealNet: Seeing Behind Objects in RGB-D Scans

• 2020 年 4 月 10 日
• 筆記

点云PCL免费知识星球，点云论文速读。

标题：RevealNet: Seeing Behind Objects in RGB-D Scans

作者：Ji Hou Angela Dai Matthias Nießner

来源：cvpr2020.

星球ID：Lionheart|点云配准

欢迎各位加入免费知识星球，获取PDF论文，欢迎转发朋友圈分享快乐。

●论文摘要

一种端到端的RGB-D场景点云补全方法。在3D重建中，经常出现不能完全扫描独立目标，导致场景中几何信息的缺失，这些丢失的信息严重影响了许多应用，例如：一个机器人需要未知几何信息完成精确的目标抓取，因此我们引入了语义补全的例子：在一个不完整的RGB-D场景中，我们尝试探测独立的目标并且猜测他们完整的目标几何形状，这将为场景交互带来新的可能性，例如虚拟现实与机器人代理，我们引入RevealNet网络来完整这项任务，一个以数据为驱动的方法来探测目标并预测他们完整的几何形状，以为着将扫描的场景分解成独立的拥有语义含义的对象，RevealNet是一个端到端的3D神经网络，能够融合颜色和几何特征信息，我们的三维网络的全卷积性质使得语义实例完成的推理在大型室内环境下的三维扫描只需一个前向通道。我们表明，预先完成完整的对象几何图形可以同时改善3D检测和实例分割性能。

●论文图集

使用RecealNet网络补全场景实例，网络中使用到了图像的颜色和场景的几何信息。场景中目标被检测出来，同时预测被检测目标完整的几何形状。

RevealNet网络结构

方法实例效果展示

方法效果对比

●英文摘要

During 3D reconstruction, it is often the case that people cannot scan each individual object from all views, resulting in missing geometry in the captured scan. This missing geometry can be fundamentally limiting for many applications, e.g., a robot needs to know the unseen geometry to perform a precise grasp on an object. Thus, we introduce the task of semantic instance completion: from an incomplete RGB-D scan of a scene, we aim to detect the individual object instances and infer their complete object geometry. This will open up new possibilities for interactions with objects in a scene, for instance for virtual or robotic agents. We tackle this problem by introducing RevealNet, a new data-driven approach that jointly detects object instances and predicts their complete geometry. This enables a semantically meaningful decomposition of a scanned scene into individual, complete 3D objects, including hidden and unobserved object parts.

RevealNet is an end-to-end 3D neural network architecture that leverages joint color and geometry feature learning. The fully-convolutional nature of our 3D network enables efficient inference of semantic instance completion for 3D scans at scale of large indoor environments in a single forward pass. We show that pre dicting complete object geometry improves both 3D detection and instance segmentation performance. We evaluate on both real and synthetic scan benchmark data for the new task, where we outperform state-of-the-art approaches by over 15 in [email protected] on ScanNet, and over 18 in [email protected] on SUNCG