- tags
- GPT, Transformers, NLP
- blog post
- OpenAI blog post
Architecture
ChatGPT takes a GPT3.5 (aka GPT3 Davinci-003) pretrained model and uses RLHF to fine-tune the model similarly to InstructGPT but with some differences in the data collection. It is also more than “just” a model since it includes extensions for Memory Store and retrieval similar to BlenderBot 3.
Parameter count
175B