Skip Connections and MLPs: The Secret Sauce in Attention
The paper says that those pesky skip connections and MLP layers that go along with attention layers is doing something very important. https://x.com/_akhaliq/status/1368739128028893188
The paper says that those pesky skip connections and MLP layers that go along with attention layers is doing something very important. https://x.com/_akhaliq/status/1368739128028893188