Inverting the Imaging Process by Learning an Implicit Camera Model

Figure 1. Our method solves inverse imaging tasks by learning an implicit neural camera model. The proposed framework consists of (a) a scene model representing scene contents and (b) a camera model simulating the camera imaging process. Given a pixel position p (2D pixel coordinate + 1D image index) at an image stack, the scene model maps it to corresponding irradiance value r (i.e., r = f (p)), and then the camera model maps the irradiance r to a pixel intensity c (i.e., c = g(r)). Two models are trained per scene and jointly optimized under the supervision of (c) a set of images captured with different camera settings (multi-focus and multi-exposure). After training, the scene irradiance have been implicitly encoded into the scene model (under an indirect supervision from the multi-setting images). We then remove the camera model and the scene model can render (d) all-in-focus HDR images by taking pixel positions as input.