Random Success: When Sampling Weights Generalizes
·47 words·1 min
Cool theoretical finding:
If you randomly sample weights until it perfectly fits your training data, those weights will also generalize well 🙀
The way I’m thinking about this: When random sampling, probability of finding small set of good weights that fits data is far more… https://x.com/gon_buzaglo/status/1756985928109682936