BigBird Finally Gets Adopted in Transformer Land
Big Bird had been the least favorite in transformer land and so many authors have refused to adopt it despite of opportunity (hello MegatronLM, T-NLG, BERT-Large). But finally some kind authors have came forward and Big Bird is no longer available: https://arxiv.org/abs/2007.14062