An Emoji in the Title? Mixers and Patch Embeddings Take Over
For this paper, my first reaction was: is that an emoji in the title? On serious note, this is very nice paper. It turns out “mixer” pattern and patch embeddings are the key ingredient, not the MLPs or attention. This rapid progress is phenomenal! https://openreview.net/forum?id=TVHS5Y4dNvM