Lune

ICML2025Top-tier venue

Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models

Congcong Zhu, Xiaoyan Xu, Jiayue Han, Jingrun Chen

Abstract

Auto-regressive partial differential equation (PDE) foundation models have shown great potential in handling time-dependent data. However, these models suffer from error accumulation caused by the shortcut problem deeply rooted in auto-regressive prediction. The challenge becomes particularly evident for out-of-distribution data, as the pretraining performance may approach random model initialization for downstream tasks with long-term dynamics. To deal with this problem, we propose physics-informed temporal alignment (PITA), a self-supervised learning framework inspired by inverse problem solving. Specifically, PITA aligns the physical dynamics discovered at different time steps on each given PDE trajectory by integrating physicsinformed constraints into the self-supervision signal. The alignment is derived from observation data without relying on known physics priors, indicating strong generalization ability to out-of-distribution data. Extensive experiments show that PITA significantly enhances the accuracy and robustness of existing foundation models on diverse time-dependent PDE data. The code is available at https://github.com/ SCAILab-USTC/PITA.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior cites Fourier Neural Operator for Parametric Partial Differential Equations
  • ComPhy: Composing Physical Models with end-to-end Alignment cites Fourier Neural Operator for Parametric Partial Differential Equations
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE TransformersZhao Yanshun, Xiaoyu Peng, Jiamin Jiang, Congcong Zhu et al.ICML 2026
  2. ComPhy: Composing Physical Models with end-to-end AlignmentAlessandro Trenta, Andrea Cossu, Davide BacciuICLR 2026

Builds on12

  1. Fourier Neural Operator for Parametric Partial Differential EquationsZongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu et al.ICLR 2021 · 3,911 citations
  2. Model Sparsity Can Simplify Machine UnlearningJinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao et al.NeurIPS 2023 · 293 citations
  3. Can contrastive learning avoid shortcut solutions?Joshua Robinson, Li Sun, Ke Yu, Kayhan Batmanghelich et al.NeurIPS 2021 · 185 citations
  4. Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer BehaviorShashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji et al.NeurIPS 2023 · 173 citations
  5. Rectifying the Shortcut Learning of Background for Few-Shot LearningXu Luo, Longhui Wei, Liangjian Wen, Jinrong Yang et al.NeurIPS 2021 · 110 citations

Related papers

  1. OmniArch: Building Foundation Model for Scientific ComputingTianyu Chen, Haoyi Zhou, Ying Li, Hao Wang et al.ICML 2025
  2. Poseidon: Efficient Foundation Models for PDEsMaximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli et al.NeurIPS 2024 · 235 citations
  3. PINFDiT: Energy-Based Physics-Informed Diffusion Transformers for General-purpose Time Series TasksDefu Cao, Wen Ye, Yizhou Zhang, Sam Griesemer et al.ICLR 2026
  4. DPM: A Novel Training Method for Physics-Informed Neural Networks in ExtrapolationJungeun Kim, Kookjin Lee, Dongeun Lee, Sheo Yon Jhin et al.AAAI 2021 · 112 citations
  5. DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic predictionRudy Morel, Jiequn Han, Edouard OyallonICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines