Tactile-Augmented Radiance Fields

Touch Pred. Vision Touch Sample Vision Figure 1. Tactile-augmented radiance fields. We capture a tactile-augmented radiance field (TaRF) from photos and sparsely sampled touch probes. To do this, we register the captured visual and tactile signals into a shared 3D space, then train a diffusion model to impute touch at other locations within the scene. Here, we visualize two touch probes and their (color coded) 3D positions in the scene. We also show two touch signals estimated by the diffusion model. The touch signals were collected using a vision-based touch sensor [32] that represents the touch signals as images. Please see our project page for video results.