- tags
- Data representation, Neural networks
- resources
- Sitzmann’s Awesome Implicit Neural Representations github page
Implicit neural representations is about parameterizing a continuous differentiable signal with a neural network. The signal is encoded within the neural network, providing a possibly more compact representation or allowing smooth parameter-based manipulation of that signal. This is a type of regression problem.
Applications of these learned representations range from simple compression, to 3D scene reconstruction from 2D images, super-resolution, semantic information inference, etc.
CPPN is an early example of a implicit neural representation implementation mainly used for pattern generation . It uses a neural network to generate patterns parameterized by two (or more) coordinates.
Usually a coordinate system is used as input for the network which will attempt to produce the signal’s value at each coordinate.
Implicit neural representations for high frequency data
To encode potentially high frequency data such as sound or images, it is much more efficient to start from periodic feature transformations. This can be achieved with periodic activation functions such as sinusoidal representation networks or SIRENs (Sitzmann et al. 2020) or by using a Fourier feature mapping (Tancik et al. 2020) .
Neural radiance fields
Neural radiance fields (NeRF) are a proposed positional encoding to better fit high-frequency data (Mildenhall et al. 2020).
A Gaussian activated radiance field with good extrapolation results: (Chng et al. 2022) (see a summary and implementation here). The idea of Gaussian activation for radiance fields was first explored in (Ramasinghe, Lucey 2022).
Bibliography
- Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein. . "Implicit Neural Representations with Periodic Activation Functions". Arxiv:2006.09661 [cs, Eess]. http://arxiv.org/abs/2006.09661.
- Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng. . "Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains". Arxiv:2006.10739 [cs]. http://arxiv.org/abs/2006.10739.
- Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng. . "Nerf: Representing Scenes as Neural Radiance Fields for View Synthesis". Arxiv:2003.08934 [cs]. http://arxiv.org/abs/2003.08934.
- Shin-Fang Chng, Sameera Ramasinghe, Jamie Sherrah, Simon Lucey. . "GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimation". Arxiv:2204.05735 [cs]. http://arxiv.org/abs/2204.05735. See notes
- Sameera Ramasinghe, Simon Lucey. . "Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-mlps". Arxiv:2111.15135 [cs]. http://arxiv.org/abs/2111.15135.