Core Context Aware Transformers for Long Context Language Modeling

Yaofo Chen, Zeng You, Shuhai Zhang, Haokun Li, Yirui Li, Yaowei Wang, Mingkui Tan

摘要

Outline 1. Background 2. Core Context Aware Attention 3. Experimental Results Conclusion