Skip the Skips: 500-Layer Neural Nets Made Simple
Huh??!! “we can therefore train a “vanilla” fully connected network and convolutional neural network—no skip connections, batch normalization, dropout, or any other architectural tweak—with 500 layers by simply adding the batch-entropy regularization term to the loss function.” https://x.com/_arohan_/status/1559709611313094656