ACL Findings2024
On the Relationship Between RNN Hidden-State Vectors and Semantic Structures
Edi Muskardin, Martin Tappler, Ingo Pill, Bernhard K. Aichernig, Thomas Pock
Abstract
We examine the assumption that hidden-state vectors of recurrent neural networks (RNNs) tend to form clusters of semantically similar vectors, which we dub the clustering hypothesis. While this hypothesis has been assumed in RNN analyses in recent years, its validity has not been studied thoroughly on modern RNN architectures. We first consider RNNs that were trained to recognize regular languages. This enables us to draw on perfect ground-truth automata in our evaluation, against which we can compare the RNN's accuracy and the distribution of the hidden-state vectors. Then, we consider context-free languages to examine if RNN states form clusters for more expressive languages. For our analysis, we fit (generalized) linear models to classify RNN states into automata states and we apply different unsupervised clustering techniques. With a new ambiguity score, derived from information entropy, we measure how well an abstraction function maps the hidden state vectors to abstract clusters. Our evaluation supports the validity of the clustering hypothesis for regular languages, especially if RNNs are well-trained, i.e., clustering techniques succeed in finding clusters of similar state vectors. However, the clustering accuracy decreases substantially for contextfree languages. This suggests that clustering is not a reliable abstraction technique for RNNs used in tasks like natural language processing.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on2
Related papers
- Decision-Guided Weighted Automata Extraction from Recurrent Neural NetworksXiyue Zhang, Xiaoning Du, Xiaofei Xie, Lei Ma et al.AAAI 2021 · 25 citations
- AdaAX: Explaining Recurrent Neural Networks by Learning Automata with Adaptive StatesDat Hong, Alberto Maria Segre, Tong WangKDD 2022 · 3 citations
- The Surprising Computational Power of Nondeterministic Stack RNNsBrian DuSell, David ChiangICLR 2023
- Recurrent Neural Language Models as Probabilistic Finite-state AutomataAnej Svete, Ryan CotterellEMNLP 2023 · 1 citation
- What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular LanguagesNadav Borenstein, Anej Svete, Robin Chan, Josef Valvoda et al.ACL 2024
