NeuralSketch2Surf: Fast Neural Surfacing of Unoriented 3D Sketches

With recent advances in VR, 3D sketches have emerged as a powerful medium for 3D model creation. However, while they already provide the user with a good perception of the intended shape, they must be surfaced before any reuse in a downstream application. This remains a challenge when sketches are unoriented, i.e., when they are simply sets of unsorted 3D strokes, without any additional normal information. We introduce NeuralSketch2Surf, the first fast and robust neural surfacing solution that processes arbitrarily unoriented sketches at interactive rates. Our approach uses S2V-Net, a new transformer network designed to mesh 3D sketches. Instead of directly inferring complex functions to represent shapes, we focus on predicting an occupancy grid, then refined using a custom smoothing function to create the desired surface. Thanks to a lightweight architecture that enables fast predictions, our method produces results in less than 2 seconds, in contrast to SOTA techniques that can take minutes or even hours. Extensive evaluations demonstrate that our method is not only fast but also generates closed surfaces with high geometric, topological, and perceptual accuracy.