Brains Predict Hierarchies—LLMs Should Too!
·38 words·1 min
This is very interesting study showing that brain predicts hierarchical representations over multiple timescales. Authors argue that LLMs should try to predict similar representation instead of just next word.
Big hint towards rethinking naive autoregressive objective… https://x.com/AIatMeta/status/1638912735143419904