Path-space differentiable rendering
Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, Shuang Zhao
Abstract
Physics-based differentiable rendering, the estimation of derivatives of radiometric measures with respect to arbitrary scene parameters, has a diverse array of applications from solving analysis-by-synthesis problems to training machine learning pipelines incorporating forward rendering processes. Unfortunately, general-purpose differentiable rendering remains challenging due to the lack of efficient estimators as well as the need to identify and handle complex discontinuities such as visibility boundaries. In this paper, we show how path integrals can be differentiated with respect to arbitrary differentiable changes of a scene. We provide a detailed theoretical analysis of this process and establish new differentiable rendering formulations based on the resulting differential path integrals. Our path-space differentiable rendering formulation allows the design of new Monte Carlo estimators that offer significantly better efficiency than state-of-the-art methods in handling complex geometric discontinuities and light transport phenomena such as caustics. We validate our method by comparing our derivative estimates to those generated using the finite-difference method. To demonstrate the effectiveness of our technique, we compare inverse-rendering performance with a few state-of-the-art differentiable rendering methods.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
39 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising cites DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer
- Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising cites Monte Carlo estimators for differential light transport
- DR.JIT: a just-in-time compiler for differentiable rendering cites Monte Carlo estimators for differential light transport
Cited by top-tier papers39
- Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and DenoisingJon Hasselgren, Nikolai Hofmann, Jacob MunkbergNeurIPS 2022 · 234 citations
- DR.JIT: a just-in-time compiler for differentiable renderingWenzel Jakob, Sébastien Speierer, Nicolas Roussel, Delio ViciniSIGGRAPH 2022 · 160 citations
- DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable RendererWenzheng Chen, Joey Litalien, Jun Gao, Zian Wang et al.NeurIPS 2021 · 74 citations
- Monte Carlo estimators for differential light transportTizian Zeltner, Sébastien Speierer, Iliyan Georgiev, Wenzel JakobSIGGRAPH 2021 · 70 citations
- Neural-PBIR Reconstruction of Shape, Material, and IlluminationCheng Sun, Guangyan Cai, Zhengqin Li, Kai Yan et al.ICCV 2023 · 56 citations
Related papers
- Path-space differentiable rendering of participating mediaCheng Zhang, Zihan Yu, Shuang ZhaoSIGGRAPH 2021 · 50 citations
- Path-Space Differentiable Rendering of Implicit SurfacesSiwei Zhou, Youngha Chang, Nobuhiko Mukai, Hiroaki Santo et al.SIGGRAPH 2024 · 1 citation
- Plateau-Reduced Differentiable Path TracingMichael Fischer, Tobias RitschelCVPR 2023
- Efficient estimation of boundary integrals for path-space differentiable renderingKai Yan, Christoph Lassner, Brian Budge, Zhao Dong et al.SIGGRAPH 2022 · 34 citations
- Robust Computation of Boundary Path Integrals Using Kernel-Density EstimationPeiyu Xu, Lifan Wu, Benedikt Bitterli, Ravi Ramamoorthi et al.SIGGRAPH 2026
