No ProbLLaMA: GPT-3 Rival Runs on Single GPU
You can now grab GPT-3 competitive model that runs on single GPU from LLaMA collection! Unlike previous OPT/BLOOM/GPT-NeoX attempts, LLaMA models are competitive with PaLM/GPT at compute optimal sizes while incorporating latest arch enhancements. 1/2 https://x.com/ylecun/status/1629189925089296386
The interesting thing about this effort is that they managed to train models only using open datasets. This at least means that foundation models won’t likely require proprietary data. This should be a big relief to many people worried about data leaks and infringement issues.