The Evolved Transformer
01 Feb 2019, Prathyush SPGoal is to apply NAS to search for a better alternative to the Transformer. We first construct a large search space inspired by the recent advances in feed-forward sequence models and then run evolutionary architecture search with warm starting by seeding our initial population with the Transformer. T
For more details, visit the source.