Lune

Profiles2021-2023

Yi Tay

cid:93) denotes technical research contributors. (cid:91) denotes data &

18Papers
3Last author
17Unique co-authors
9First author
0Solo

First and last author come from byline position, which some venues assign alphabetically.

Recent papers18

  1. UL2: Unifying Language Learning ParadigmsAuthor 1 of 14ICLR 2023 · 97 citations
  2. Transformer Memory as a Differentiable Search IndexAuthor 1 of 13NeurIPS 2022 · 506 citations
  3. ExT5: Towards Extreme Multi-Task Scaling for Transfer LearningAuthor 2 of 16ICLR 2022 · 237 citations
  4. Scarf: Self-Supervised Contrastive Learning using Random Feature CorruptionAuthor 3 of 4ICLR 2022 · 233 citations
  5. Charformer: Fast Character Transformers via Gradient-based Subword TokenizationAuthor 1 of 10ICLR 2022 · 198 citations
  6. The Efficiency MisnomerAuthor 5 of 5ICLR 2022 · 116 citations
  7. HyperPrompt: Prompt-based Task-Conditioning of TransformersAuthor 4 of 13ICML 2022 · 110 citations
  8. Scale Efficiently: Insights from Pretraining and Finetuning TransformersAuthor 1 of 10ICLR 2022 · 67 citations
  9. Improving Compositional Generalization with Self-Training for Data-to-Text GenerationAuthor 3 of 6ACL 2022 · 34 citations
  10. SCENIC: A JAX Library for Computer Vision Research and BeyondAuthor 4 of 4CVPR 2022 · 30 citations
  11. Sharpness-Aware Minimization Improves Language Model GeneralizationAuthor 3 of 3ACL 2022
  12. Long Range Arena : A Benchmark for Efficient TransformersAuthor 1 of 10ICLR 2021 · 881 citations
  13. Synthesizer: Rethinking Self-Attention for Transformer ModelsAuthor 1 of 6ICML 2021 · 399 citations
  14. HyperGrid Transformers: Towards A Single Model for Multiple TasksAuthor 1 of 5ICLR 2021 · 44 citations
  15. OmniNet: Omnidirectional Representations from TransformersAuthor 1 of 9ICML 2021 · 33 citations
  16. How Reliable are Model Diagnostics?Author 2 of 3ACL Findings 2021 · 15 citations
  17. Are Pretrained Convolutions Better than Pretrained Transformers?Author 1 of 7ACL 2021
  18. StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language ModelingAuthor 2 of 6ACL 2021
Dusk over the sea between two cliffs drawn in fine vertical lines