- tags
- Machine learning
Two-layers neural network
Mathematically, a simple two-layers neural network with relu non-linearities can be written like below. For an input vector \(x \in \mathbb{R}^D\), \(\mathbf{a} = (a_1, \cdots, a_N)\in \mathbb{R}^M\) are the output weights, \(\mathbf{b} = (b_1, \cdots, b_N)\in \mathbb{R}^D\) are the input weights
\[ h(x) = \frac{1}{m} \sum_{i=1}^m a_i \max\{ b_i^\top x,0\}, \]
Backlinks
- Neural network pruning
- Attention
- Adaptive Computation Time
- Convolutional neural networks
- Neural architecture search
- Data representation
- Hopfield Networks
- Cellular neural networks
- Gradient descent for wide two-layer neural networks – I : Global convergence
- Implicit neural representations
- Generative adversarial networks
- Talk: Artificial Intelligence: A Guide for Thinking Humans
- Adversarial examples
- Recurrent neural networks
- Transformers
- Notes on: Adapting to Unseen Environments through Explicit Representation of Context by Tutum, C., & Miikkulainen, R. (2020)
- Notes on: Molecule Attention Transformer by Maziarka, Ł., Danel, T., Mucha, S., Rataj, K., Tabor, J., & Jastrzębski, S. (2020)
- Notes on: Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data by Bender, E. M., & Koller, A. (2020)
- Notes on: Evolving Neural Networks through Augmenting Topologies by Stanley, K. O., & Miikkulainen, R. (2002)
- Compression
- Variational autoencoders
- Mean field theory of neural networks (talk)
- Neural network training
- Autoencoders
- The Lottery ticket hypothesis
- Talk: The Importance of Open-Endedness in AI and Machine Learning
- CPPN