Digit Dilemma: Tokenizing Numbers Boosts Model Math
·44 words·1 min
Very interesting insight: Tokenizing numbers into individual digits significantly improves arithmetic ability of the model! The impacts is so large that it makes or breaks it.
We still have very little idea on best tokenization strategy. My intuition is that ideal tokenization… https://x.com/rasbt/status/1661754946625105920