Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data
Praveen Venkatesh, Corbett Bennett, Sam Gale, Tamina K. Ramirez, Greggory Heller, Severine Durand, Shawn R. Olsen, Stefan Mihalas
Abstract
Recent advances in neuroscientific experimental techniques have enabled us to simultaneously record the activity of thousands of neurons across multiple brain regions. This has led to a growing need for computational tools capable of analyzing how task-relevant information is represented and communicated between several brain regions. Partial information decompositions (PIDs) have emerged as one such tool, quantifying how much unique, redundant and synergistic information two or more brain regions carry about a task-relevant message. However, computing PIDs is computationally challenging in practice, and statistical issues such as the bias and variance of estimates remain largely unexplored. In this paper, we propose a new method for efficiently computing and estimating a PID definition on multivariate Gaussian distributions. We show empirically that our method satisfies an intuitive additivity property, and recovers the ground truth in a battery of canonical examples, even at high dimensionality. We also propose and evaluate, for the first time, a method to correct the bias in PID estimates at finite sample sizes. Finally, we demonstrate that our Gaussian PID effectively characterizes inter-areal interactions in the mouse brain, revealing higher redundancy between visual areas when a stimulus is behaviorally relevant.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext c5572479-8473-481e-81fe-2c0368d74997Cited by top-tier papers5
- Partial Information Decomposition via Normalizing Flows in Latent Gaussian DistributionsWenyuan Zhao, Adithya Balachandran, Chao Tian, Paul Pu LiangNeurIPS 2025 · 5 citations
- Analytically deriving Partial Information Decomposition for affine systems of stable and convolution-closed distributionsChaitanya Goswami, Amanda MerkleyNeurIPS 2024 · 5 citations
- SΩI: Score-based O-INFORMATION EstimationMustapha Bounoua, Giulio Franzese, Pietro MichiardiICML 2024 · 3 citations
- Building Massively Multimodal Foundation Models with Interaction-aware Mixture-of-ExpertsXing Han, Hsing-Huan Chung, Joydeep Ghosh, Paul Pu Liang et al.ICLR 2026 · 1 citation
- Efficient Quantification of Multimodal Interaction at Sample LevelZequn Yang, Hongfa Wang, Di HuICML 2025
Builds on2
- Estimating the Unique Information of Continuous VariablesAri Pakman, Amin Nejatbakhsh, Dar Gilboa, Abdullah Makkeh et al.NeurIPS 2021 · 43 citations
- An Information-Theoretic Quantification of Discrimination with Exempt FeaturesSanghamitra Dutta, Praveen Venkatesh, Piotr Mardziel, Anupam Datta et al.AAAI 2020 · 34 citations
Related papers
- Quantifying & Modeling Multimodal Interactions: An Information Decomposition FrameworkPaul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling et al.NeurIPS 2023 · 120 citations
- What should a neuron aim for? Designing local objective functions based on information theoryAndreas Christian Schneider, Valentin Neuhaus, David Alexander Ehrlich, Abdullah Makkeh et al.ICLR 2025
- Demixed shared component analysis of neural population data from multiple brain areasYu Takagi, Steven W. Kennerley, Jun-ichiro Hirayama, Laurence T. HuntNeurIPS 2020 · 2 citations
- Disentanglement Analysis with Partial Information DecompositionSeiya Tokui, Issei SatoICLR 2022 · 16 citations
- Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information DecompositionWanlong Fang, Tianle Zhang, Wen Tao, Alvin ChanICML 2026 · 17 citations
