Neural networks

tags
Machine learning

Two-layers neural network

Mathematically, a simple two-layers neural network with relu non-linearities can be written like below. For an input vector $$x \in \mathbb{R}^D$$, $$\mathbf{a} = (a_1, \cdots, a_N)\in \mathbb{R}^M$$ are the output weights, $$\mathbf{b} = (b_1, \cdots, b_N)\in \mathbb{R}^D$$ are the input weights

$h(x) = \frac{1}{m} \sum_{i=1}^m a_i \max\{ b_i^\top x,0\},$

← Back to Notes