Batch Norms Strike Again: Fine-Tuning CNNs by Training Only Batch Norms
A surprising paper last year was only training batch norms and still getting good accuracy on CNNs (https://arxiv.org/abs/2003.00152). In below paper, now authors demonstrate that for fine tuning if you only fine tune batch norms (leave other weights as-is) then you get similar perf! https://x.com/deep_rl/status/1359776792765214723