When 100X Larger Models Aren't Enough
·34 words·1 min
Inference compute scaling is the most fertile and rewarding area of research. The hard problems require too many FLOPS than are possible from a single/few forward pass on even 100X larger models. https://x.com/Azaliamirh/status/1819077194385445302