Lune

Profiles2021-2024

Mostafa Dehghani

Google Deepmind

21Papers
3Last author
39Unique co-authors
3First author
0Solo

First and last author come from byline position, which some venues assign alphabetically.

Recent papers21

  1. Fractal Patterns May Illuminate the Success of Next-Token PredictionAuthor 3 of 3NeurIPS 2024 · 8 citations
  2. The Impact of Group Membership Bias on the Quality and Fairness of Exposure in RankingAuthor 4 of 4SIGIR 2024 · 2 citations
  3. Scaling Vision Transformers to 22 Billion ParametersAuthor 1 of 42ICML 2023 · 848 citations
  4. UL2: Unifying Language Learning ParadigmsAuthor 2 of 14ICLR 2023 · 97 citations
  5. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?Author 2 of 10EMNLP Findings 2023 · 34 citations
  6. Adaptive Computation with Elastic Input SequenceAuthor 5 of 6ICML 2023 · 27 citations
  7. Transcending Scaling Laws with 0.1% Extra ComputeAuthor 17 of 17EMNLP 2023 · 27 citations
  8. Λ\Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among CellsAuthor 5 of 7ICLR 2023
  9. Transformer Memory as a Differentiable Search IndexAuthor 3 of 13NeurIPS 2022 · 506 citations
  10. Exploring the Limits of Large Scale Pre-trainingAuthor 2 of 4ICLR 2022 · 135 citations
  11. The Efficiency MisnomerAuthor 1 of 5ICLR 2022 · 116 citations
  12. Scale Efficiently: Insights from Pretraining and Finetuning TransformersAuthor 2 of 10ICLR 2022 · 67 citations
  13. Discrete Representations Strengthen Vision Transformer RobustnessAuthor 3 of 6ICLR 2022 · 47 citations
  14. SCENIC: A JAX Library for Computer Vision Research and BeyondAuthor 1 of 4CVPR 2022 · 30 citations
  15. ViViT: A Video Vision TransformerAuthor 2 of 6ICCV 2021 · 2,947 citations
  16. Long Range Arena : A Benchmark for Efficient TransformersAuthor 2 of 10ICLR 2021 · 881 citations
  17. TokenLearner: Adaptive Space-Time Tokenization for VideosAuthor 4 of 5NeurIPS 2021 · 274 citations
  18. IDF++: Analyzing and Improving Integer Discrete Flows for Lossless CompressionAuthor 3 of 5ICLR 2021 · 38 citations
  19. OmniNet: Omnidirectional Representations from TransformersAuthor 2 of 9ICML 2021 · 33 citations
  20. Are Pretrained Convolutions Better than Pretrained Transformers?Author 2 of 7ACL 2021
  21. Parameter-efficient Multi-task Fine-tuning for Transformers via Shared HypernetworksAuthor 3 of 4ACL 2021
Dusk over the sea between two cliffs drawn in fine vertical lines