ACL2023

How does the brain process syntactic structure while listening?

Subba Reddy Oota, Mounika Marreddy, Manish Gupta, Raju S. Bapi

被引用 3 次

摘要

Syntactic parsing is the task of assigning a syntactic structure to a sentence. There are two popular syntactic parsing methods: constituency and dependency parsing. Recent works have used syntactic embeddings based on constituency trees, incremental top-down parsing, and other word syntactic features for brain activity prediction given the text stimuli to study how the syntax structure is represented in the brain's language network. However, the effectiveness of dependency parse trees or the relative predictive power of the various syntax parsers across brain areas, especially for the listening task, is yet unexplored. In this study, we investigate the predictive power of the brain encoding models in three settings: (i) individual performance of the constituency and dependency syntactic parsing based embedding methods, (ii) efficacy of these syntactic parsing based embedding methods when controlling for basic syntactic signals, (iii) relative effectiveness of each of the syntactic embedding methods when controlling for the other. Further, we explore the relative importance of syntactic information (from these syntactic embedding methods) versus semantic information using BERT embeddings. We find that constituency parsers help explain activations in the temporal lobe and middle-frontal gyrus, while dependency parsers better encode syntactic structure in the angular gyrus and posterior cingulate cortex. Although semantic signals from BERT are more effective compared to any of the syntactic features or embedding methods, syntactic embedding methods explain additional variance for a few brain regions. We make our code publicly available 1 . 118 hension effort (Friederici, 2011) using carefully de-119 signed sentence/phrase stimuli. In the past decade, 120 the study of syntactic processing has been extended 121 to naturalistic settings that use narratives, such as 122 reading (Reddy and Wehbe, 2021) or listening to 123 stories (Bhattasali et al., 2018; Zhang et al., 2022) 124 generally in a task-free setting. Due to the com-125 plexity of extracting syntactic word embeddings 126 from sentence parsers, investigation of the predic-127 tive power of sentence parsers for brain encoding, 128 especially for the neuroimaging data from natural-129 istic listening paradigms is still under-explored. 130 Brain Regions of Interest (ROIs) for syntactic 131 processing: Several classical studies report the 132 involvement of a language network of mostly left-133 lateralised cortical regions including the left in-134 ferior frontal gyrus (IFG) with sub regions (BA 135 44 and BA 45), the left posterior superior tempo-136 ral gyrus (pSTG), and the left anterior temporal 137