Songyou Peng - Shape As Points

Talking Papers Podcast

In this episode of the Talking Papers Podcast, I hosted Songyou Peng to chat about his paper “Shape As Points: A Differentiable Poisson Solver”, published in NeurIPS 2021. In this paper, they take on the task of surface reconstruction and propose a hybrid representation that unifies explicit and implicit representation in addition to a differentiable solver for the classic Poisson surface reconstruction. I have been following Songyou's work for a while and was very surprised to discover that he is just about midway through his PhD (with so many good papers, I thought he is about to finish!). We first met online at the ICCV 2021 workshop on "Learning 3D Representations for Shape and Appearance" and I immediately flagged him as one of the next guests on the podcast.
It was a pleasure recording this episode with him.
AUTHORS

Songyou Peng, Chiyu Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, Andreas Geiger
ABSTRACT

In recent years, neural implicit representations gained popularity in 3D reconstruction due to their expressiveness and flexibility. However, the implicit nature of neural implicit representations results in slow inference time and requires careful initialization. In this paper, we revisit the classic yet ubiquitous point cloud representation and introduce a differentiable point-to-mesh layer using a differentiable formulation of Poisson Surface Reconstruction (PSR) that allows for a GPU-accelerated fast solution of the indicator function given an oriented point cloud. The differentiable PSR layer allows us to efficiently and differentiably bridge the explicit 3D point representation with the 3D mesh via the implicit indicator field, enabling end-to-end optimization of surface reconstruction metrics such as Chamfer distance. This duality between points and meshes hence allows us to represent shapes as oriented point clouds, which are explicit, lightweight and expressive. Compared to neural implicit representations, our Shape-As-Points (SAP) model is more interpretable, lightweight, and accelerates inference time by one order of magnitude. Compared to other explicit representations such as points, patches, and meshes, SAP produces topology-agnostic, watertight manifold surfaces. We demonstrate the effectiveness of SAP on the task of surface reconstruction from unoriented point clouds and learning-based reconstruction.

RELATED PAPERS

📚 Poisson Surface Reconstruction

📚 Occupancy Networks

📚 Convolutional Occupancy Networks

LINKS AND RESOURCES

💻 Project Page: https://pengsongyou.github.io/sap

💻 CODE: https://github.com/autonomousvision/shape_as_points

📚 Paper

🤐Paper's peer review

To stay up to date with Songyou's latest research, check out his personal page and follow him on:
👨‍🎓 Google Scholar
🐦Twitter
👨‍🎓LinkedIn
CONTACT

If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via

🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

📧Subscribe to our mailing list: http://eepurl.com/hRznqb

🐦Follow us on Twitter: https://twitter.com/talking_papers

🎥YouTube Channel: https://bit.ly/3eQOgwP