Skip to main content

When GPT-3 Met Code: The Birth of ChatGPT's Superpowers

·48 words·1 min · Download pdf

This is a great post on what changed from GPT3 to ChatGPT models. The best guess is that two most striking core abilities, (1) zero shot task generalization already existed in GPT3 but suddenly unlocked by scaling instruction tuning, (2) CoT is from training on code!! https://x.com/Francis_YAO_/status/1602213927102066688

Discussion