- tags
- Applied maths
The Softmax can refer to two mathematical functions:
- In machine learning a softmax is the function which normalizes a vector of values to a probability vector: \(\text{softmax}(\mathbf{x}) = \dfrac{e^{\mathbf{x}}}{\sum_i e^{x_i}}\) where \(\mathbf{x} = (x_i) \in \mathbb{R}^n\). This function could also be called soft-argmax because it is a smooth approximation of the discrete argmax function.
- It may also refer to a smoothed maximum function like \(\epsilon \log \sum_i \exp (x_i / \epsilon)\) which approximates the \(\text{max}\) function in the limit \(\epsilon \rightarrow 0\)