- tags
- Machine learning, Language
NLP is about creating algorithms that can manipulate and use language. It is often thought that having functioning NLP algorithms that provably “understand” language would be equivalent to reaching human-level Artificial Intelligence.
Tasks
Language modeling
Text classification
Question answering
Data manipulation
There are several ways to encode text data.
- One-hot encoding of characters
- One-hot encoding of words
- Byte-pair encoding which can be seen as being a compromise between the two