Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion

Figure 1 . Our framework, Uni-renderer, empowers the generative model to function both as a renderer and an inverse renderer by approximating the rendering equation using a data-driven approach. Given intrinsic attributes, Uni-renderer generates photo-realistic images, functioning as a renderer. When provided with a single RGB image, it effectively decomposes the intrinsic properties, functioning as an inverse renderer. Top: Uni-renderer generates smooth variations as renderer. Setting the roughness value to 1.0 results in the "dog" case, shown at the top, lacking specular highlights. Conversely, setting the metallic value to 1 makes the "hat" case appear metallic. Bottom Left: When functioning as an inverse renderer, Uni-renderer decomposes the intrinsic properties of a single RGB image. Bottom Right: Uni-renderer generates relighting results under different environment lighitngs.