- tags
- Neural networks
Transformers are a neural network architecture based on a mechanism called Attention.
They have been particularly successful for NLP applications.
Transformers
Links to this note
- Language modeling
- Notes on: Information-Theoretic Probing with Minimum Description Length by Voita, E., & Titov, I. (2020)
- Notes on: The geometry of integration in text classification RNNs by Aitken, K., Ramasesh, V. V., Garg, A., Cao, Y., Sussillo, D., & Maheswaranathan, N. (2020)
- Notes on: Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention by Katharopoulos, A., Vyas, A., Pappas, N., & Fleuret, F. (2020)