Lune

NeurIPS2024Top-tier venue

Analytically deriving Partial Information Decomposition for affine systems of stable and convolution-closed distributions

Chaitanya Goswami, Amanda Merkley

Abstract

Bivariate partial information decomposition (PID) has emerged as a promising tool for analyzing interactions in complex systems, particularly in neuroscience. PID achieves this by decomposing the information that two sources (e.g., different brain regions) have about a target (e.g., a stimulus) into unique, redundant, and synergistic terms. However, the computation of PID remains a challenging problem, often involving optimization over distributions. While several works have been proposed to compute PID terms numerically, there is a surprising dearth of work on computing PID terms analytically. The only known analytical PID result is for jointly Gaussian distributions. In this work, we present two theoretical advances that enable analytical calculation of the PID terms for numerous wellknown distributions, including distributions relevant to neuroscience, such as Poisson, Cauchy, and binomial. Our first result generalizes the analytical Gaussian PID result to the much larger class of stable distributions. We also discover a theoretical link between PID and the emerging fields of data thinning and data fission. Our second result utilizes this link to derive analytical PID terms for two more classes of distributions: convolution-closed distributions and a sub-class of the exponential family. Furthermore, we provide an analytical upper bound for approximately computing PID for convolution-closed distributions, whose tightness we demonstrate in simulation.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data cites An Information-Theoretic Quantification of Discrimination with Exempt Features
  • Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data cites Estimating the Unique Information of Continuous Variables
  • Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework cites An Information-Theoretic Quantification of Discrimination with Exempt Features
  • Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework cites Estimating the Unique Information of Continuous Variables
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on6

  1. Quantifying & Modeling Multimodal Interactions: An Information Decomposition FrameworkPaul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling et al.NeurIPS 2023 · 120 citations
  2. Estimating the Unique Information of Continuous VariablesAri Pakman, Amin Nejatbakhsh, Dar Gilboa, Abdullah Makkeh et al.NeurIPS 2021 · 43 citations
  3. An Information-Theoretic Quantification of Discrimination with Exempt FeaturesSanghamitra Dutta, Praveen Venkatesh, Piotr Mardziel, Anupam Datta et al.AAAI 2020 · 34 citations
  4. Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional DataPraveen Venkatesh, Corbett Bennett, Sam Gale, Tamina K. Ramirez et al.NeurIPS 2023 · 28 citations
  5. Disentanglement Analysis with Partial Information DecompositionSeiya Tokui, Issei SatoICLR 2022 · 16 citations

Related papers

  1. Partial Information Decomposition via Normalizing Flows in Latent Gaussian DistributionsWenyuan Zhao, Adithya Balachandran, Chao Tian, Paul Pu LiangNeurIPS 2025 · 5 citations
  2. What should a neuron aim for? Designing local objective functions based on information theoryAndreas Christian Schneider, Valentin Neuhaus, David Alexander Ehrlich, Abdullah Makkeh et al.ICLR 2025
  3. Probabilistic Tensor Decomposition of Neural Population Spiking ActivityHugo Soulat, Sepiedeh Keshavarzi, Troy W. Margrie, Maneesh SahaniNeurIPS 2021 · 9 citations
  4. Efficient Quantification of Multimodal Interaction at Sample LevelZequn Yang, Hongfa Wang, Di HuICML 2025
  5. Uncovering motifs of concurrent signaling across multiple neuronal populationsEvren Gokcen, Anna Jasper, Alison Xu, Adam Kohn et al.NeurIPS 2023 · 27 citations
Dusk over the sea between two cliffs drawn in fine vertical lines