Deep Learning Recommenders Fall Flat: Heuristics Have the Last Laugh
·80 words·1 min
Ouch! This paper tried replicating 18 papers in the area of recommendations using deep learning but could replicate only 7. And 6 of those 7 could be outperformed with simple heuristic while 7th could be outperformed by well-tuned non-neural method. https://arxiv.org/abs/1907.06902
Even though this is a very small dataset, correlation with h5-index is inescapable!
Conference | reproducibility | h5-index KDD | 43% | 77 WWW | 33% | 70 SIGIR | 25% | 55 RecSys | 13% | 40