Machine Learning's Algorithmic Groundhog Day

ML field seems to always pass through these stages:

1. Invent breakthrough algo1
2. Claims of new algos beating algo1
3. Bag of tricks beats all new algos
4. (Many years passes)
5. Go back to 1. https://x.com/hwchung27/status/1668729544701001729

Discussion