When GPT-3 Met Code: The Birth of ChatGPT's Superpowers
This is a great post on what changed from GPT3 to ChatGPT models. The best guess is that two most striking core abilities, (1) zero shot task generalization already existed in GPT3 but suddenly unlocked by scaling instruction tuning, (2) CoT is from training on code!! https://x.com/Francis_YAO_/status/1602213927102066688