Weaving Longer Contexts: Yarn Scaling for LLMs

Yarn scaling technique looks amazing to extend the context length of LLMs through fine tuning with no sacrifice on the base model performance! https://x.com/EnricoShippole/status/1697317625116742119

Discussion