Clock Blocked: How Dennard Scaling Stalls AI Progress
It turns out the FLOP/s growth have slowed after 2012 because clock speeds got maxed out (aka Dennard scaling). This graph allows us to extrapolate AI progress. For ex, more people regularly experimenting with 100X computationally expensive models is likely about a decade away. https://x.com/MikePFrank/status/1601463780080164865