GPT-3 Learns New Tricks On The Fly

These experiments by @MelMitchell1 are quite surreal. Given that GPT-3 training data didn’t had any of these examples (which is why 1-5 shot learning), this online generalization is truly bewildering!

https://medium.com/@melaniemitchell.me/can-gpt-3-make-analogies-16436605c446

So far I have been in camp that there is a clear line between pattern recognition and reasoning. Much of the ML and DL operates in pattern recognition land often masquerading as reasoning in narrow domain. But more I see GPT-3 results more it seems that the line is pretty blurry.

Discussion