Small But Mighty: 3B Model Outsmarts 58X Larger LLM

Instruction tuning is proving to the most important weapon to reduce size of LLMs. Last year we show T0 model beating 16X larger GPT-3 on zero shot. Below paper now pushes the limits with mere 3B model beating 58X larger OPT-IML!!

Single GPU LLMs getting closer! https://x.com/ShayneRedford/status/1620805305801261058

Discussion