GPT-2 Can't Two-Step But Jives at Five

Novelty in GPT-2 generations: It has hard-time producing novel bi-grams or tri-grams but 5-grams are often novel. https://x.com/RTomMcCoy/status/1461566108201365511

Discussion