Skip the Soap: Why Long Context Fails in ICL
·51 words·1 min
You can either twiddle your thumbs waiting for the next episode of soap or read a below good paper! ICL is one of the most important phenomena we have discovered in machine learning but the long context still doesn’t work as expected. This paper provides some insights on why. https://x.com/ZainHasan6/status/1726338773158408268