Grok and Roll: How 16 Beat GPT-3.5 in 4 Months
·50 words·1 min
In just 4 months, a team of 16 has produced a model that is probably the 2nd best SOTA, better than GPT3.5, Claud-2 and Inflection-1!
My guess is that Grok-1 is 70B model trained on 10k GPUs for 2 months. Most importantly, they also used data from http://x.com,… https://x.com/elonmusk/status/1721027243571380324