ERNIE is a series of powerful models proposed by baidu, especially in Chinese tasks, including ERNIE1.0, ERNIE2.0, ERNIE3.0, ERNIE-Gram, ERNIE-health, etc.
These models are contributed by nghuyong and the official code can be found in PaddleNLP (in PaddlePaddle).
Take ernie-1.0-base-zh as an example:
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")
model = AutoModel.from_pretrained("nghuyong/ernie-1.0-base-zh")| Model Name | Language | Description |
|---|---|---|
| ernie-1.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-base-en | English | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-large-en | English | Layer:24, Heads:16, Hidden:1024 |
| ernie-3.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-3.0-medium-zh | Chinese | Layer:6, Heads:12, Hidden:768 |
| ernie-3.0-mini-zh | Chinese | Layer:6, Heads:12, Hidden:384 |
| ernie-3.0-micro-zh | Chinese | Layer:4, Heads:12, Hidden:384 |
| ernie-3.0-nano-zh | Chinese | Layer:4, Heads:12, Hidden:312 |
| ernie-health-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-gram-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
You can find all the supported models from huggingface's model hub: huggingface.co/nghuyong, and model details from paddle's official repo: PaddleNLP and ERNIE.
[[autodoc]] ErnieConfig - all
[[autodoc]] models.ernie.modeling_ernie.ErnieForPreTrainingOutput
[[autodoc]] ErnieModel - forward
[[autodoc]] ErnieForPreTraining - forward
[[autodoc]] ErnieForCausalLM - forward
[[autodoc]] ErnieForMaskedLM - forward
[[autodoc]] ErnieForNextSentencePrediction - forward
[[autodoc]] ErnieForSequenceClassification - forward
[[autodoc]] ErnieForMultipleChoice - forward
[[autodoc]] ErnieForTokenClassification - forward
[[autodoc]] ErnieForQuestionAnswering - forward