Automatic and Structure-Aware Sparsification of Hybrid Neural ODEs with Application to Glucose Prediction
Bob Junyi Zou, Lu Tian
Abstract
Hybrid neural ordinary differential equations (neural ODEs) integrate mechanistic models with neural ODEs, offering strong inductive bias and flexibility, and are particularly advantageous in data-scarce healthcare settings. However, excessive latent states and interactions from mechanistic models can lead to training inefficiency and over-fitting, limiting practical effectiveness of hybrid neural ODEs. In response, we propose a new hybrid pipeline for automatic state selection and structure optimization in mechanistic neural ODEs, combining domain-informed graph modifications with data-driven regularization to sparsify the model for improving predictive performance and stability while retaining mechanistic plausibility. Experiments on synthetic and real-world data show improved predictive performance and robustness with desired sparsity, establishing an effective solution for hybrid model reduction in healthcare applications. 1. S = s 1 , . . . , s n is the set of state variables with cardinality n, and s i (t) : [0, +∞) → R is a real-valued function of t representing the value of state s i at time t. 2. X = x 1 , . . . , x m is the set of exogenous input variables with cardinality m, and x j (t) : [0, +∞) → R is a function of t representing the value of input x j at time t.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 6130bea5-e467-4e81-a339-8ca290d6814eBuilds on9
- Learning to Simulate Complex Physics with Graph NetworksAlvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying et al.ICML 2020 · 1,439 citations
- GraphSAINT: Graph Sampling Based Inductive Learning MethodHanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan et al.ICLR 2020 · 1,155 citations
- On the Parameterization and Initialization of Diagonal State Space ModelsAlbert Gu, Karan Goel, Ankit Gupta, Christopher RéNeurIPS 2022 · 690 citations
- Graph Structure of Neural NetworksJiaxuan You, Jure Leskovec, Kaiming He, Saining XieICML 2020 · 168 citations
- The Causal-Neural Connection: Expressiveness, Learnability, and InferenceKevin Xia, Kai-Zhan Lee, Yoshua Bengio, Elias BareinboimNeurIPS 2021 · 158 citations
Related papers
- Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease ProgressionZhaozhi Qian, William R. Zame, Lucas M. Fleuren, Paul W. G. Elbers et al.NeurIPS 2021 · 88 citations
- Hybrid2 Neural ODE Causal Modeling and an Application to Glycemic ResponseBob Junyi Zou, Matthew E. Levine, Dessi P. Zaharieva, Ramesh Johari et al.ICML 2024 · 13 citations
- Model-Based Control with Sparse Neural DynamicsZiang Liu, Genggeng Zhou, Jeff He, Tobia Marcucci et al.NeurIPS 2023 · 28 citations
- Sparse Flows: Pruning Continuous-depth ModelsLucas Liebenwein, Ramin M. Hasani, Alexander Amini, Daniela RusNeurIPS 2021 · 21 citations
- Sparsity in Continuous-Depth Neural NetworksHananeh Aliee, Till Richter, Mikhail Solonin, Ignacio Ibarra et al.NeurIPS 2022 · 21 citations
