ERNIE

tags

Transformers, BERT, NLP

paper

(Zhang et al. 2019)

Architecture

This transformer uses two stacked BERT for encoding: one for the text, one for the entities in a knowledge graph.

Parameter count

114M

Bibliography

1. Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, Qun Liu. June 4, 2019. "ERNIE: Enhanced Language Representation with Informative Entities". arXiv. DOI.