MultimodalStudio: A Heterogeneous Sensor Dataset and Framework for Neural Rendering across Multiple Imaging Modalities

https://lttm.github.io/MultimodalStudio/ Figure 1. Overview of the proposed framework. MMS-FW exploits unaligned multimodal frames acquired by sensors with different intrinsics to render perfectly aligned novel views for each modality. The mosaick pattern for each modality is shown in the top corners.