Counting the Cost: AI's Million-Dollar Computations
What is the computation cost for modern AI research papers?
My calculator says the cost of computation needed for NAS-Bench 101 paper was at least $4.7 million (120 TPU years). Similarly OpenAI Rubic cube solving computation cost was at least $3.9 million (13000 core years).