Inference Scaling
Inference compute scaling is the most fertile and rewarding area of research. The hard problems require too many FLOPS than are not possible from just a single/few forward pass on even 100X larger models. https://x.com/Azaliamirh/status/1819077194385445302