Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding

edge that is reflected in language-only tasks that require implicit visual reasoning. Our findings bear importance in the broader context of multimodal learning, providing principled guidelines for the choice of text encoders used in such contexts 1 .