An Information-Theoretic Parameter-Free Bayesian Framework for Probing Labeled Dependency Trees from Attention Score

Figuring out how neural language models comprehend syntax acts as a key to revealing how they understand languages. We systematically analyzed methods for finding syntax structures in models, namely probing, and found limitations yet widely exist in previous probing practice. We proposed a method capable of estimating mutual information (MI) and extracting dependency trees from attention scores in a mathematical-rigorous way, requiring no additional network training effort. Compared with previous approaches, it has a much simpler model, while being able to probe more complex dependency trees, also transparent for fine-grained explanation. We tested our method on several open-source LLMs and demonstrated its effectiveness by systematically comparing it with a great many competitive baselines. Several informative conclusions can be drawn by further analysis of the results, shedding light on our method’s explanatory potential. Our code is released at https://github.com/ChristLBUPT/IPBP.