It’s a great pleasure to welcome David B. Lindell and Julien Martel from Stanford Engineering to our Lab!

Abstract: Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations and new capabilities in neural rendering and view synthesis. However, conventional network architectures for such implicit neural representations are incapable of modeling signals at scale with fine detail and fail to represent derivatives and integrals of signals. In this talk, we describe three recent approaches to solve these challenging problems. First, we introduce sinusoidal representation networks or SIREN, which are ideally suited for representing complex natural signals and their derivatives. Using SIREN, we can represent images, wavefields, video, sound, and their derivatives, allowing us to solve differential equations using this type of neural network. Second, we introduce a new framework for solving integral equations using implicit neural representation networks. Our automatic integration framework, AutoInt, enables the calculation of any definite integral with two evaluations of a neural network. This allows fast inference and rendering when applied to neural rendering techniques based on volume rendering.  Finally, we introduce a new architecture and method for scaling up implicit representations, called Adaptive Coordinate Networks (ACORN). The approach relies on a hybrid implicit–explicit representation and a learned, online multiscale decomposition of the target signal. We use ACORN to demonstrate the first experiments that fit gigapixel images to nearly 40 dB peak signal-to-noise ratio (an 1000x increase in scale over previous experiments), and we reduce training times for 3D shape fitting from days to hours or minutes while improving memory requirements by over an order of magnitude.

Short Bios Julien Martel (http://www.jmartel.net/) is a Postdoctoral Research Fellow at Stanford University in the Computational Imaging Lab led by Gordon Wetzstein. His research interests are in unconventional visual sensing and processing. More specifically, his current topics of research include the co-design of hardware and algorithms for visual sensing, the design of methods for vision sensors with in-pixel computing capabilities, and the use of novel representations for visual data such as neural implicit representations.

David B. Lindell (https://davidlindell.com) is a Postdoctoral Scholar in the Department of Electrical Engineering at Stanford University. His research interests are in the areas of computational imaging, machine learning, and remote sensing. Most recently, he has worked on developing neural representations for applications in vision and rendering. He has also developed advanced 3D imaging systems to capture objects hidden around corners or through scattering media.

Register for more upcoming talks here!

References

SIREN: https://vsitzmann.github.io/siren/

ACORN: https://www.computationalimaging.org/publications/acorn/

AutoInt: https://www.computationalimaging.org/publications/automatic-integration/

This video is released under CC BY 4.0. Please feel free to share and reuse.

For reminders to watch the new video follow on Twitter or LinkedIn. Also, join our network for information about talks, videos, and job offers in our Facebook and LinkedIn Groups.

Music Reference: 
Damiano Baldoni - Thinking of You (Intro)
Damiano Baldoni - Poenia (Outro)