FabricTryOn: Taming Image Editing Models for Garment Re-Texturing

Trying different fabrics on existing garments is a widely applicable problem in digital fashion and computer graphics. A comprehensive transformation involves both material reflectance and geometric deformation from fabric drape. In this work, we focus on the visual aspects of this challenge and simplify fabric try-on to a re-texturing task that replaces garment materials while preserving the original geometry and illumination. Prior approaches perform garment re-texturing via 3D or UV-space reconstruction and rendering, making them sensitive to reconstruction accuracy and rendering fidelity. Recent diffusion-based material transfer methods either lack fine-grained geometric and material control or suffer from domain gaps due to training on synthetic rendered data. We propose a fabric try-on framework that leverages the generative priors of modern image editing models. Motivated by the in-context generation capability of Multimodal Diffusion Transformers, we reformulate garment re-texturing as a two-stage process consisting of fabric removal and fabric application via an intermediate material-normalized image. We further introduce a real-image data curation pipeline and a context-aware tile augmentation strategy, enabling coherent and photorealistic fabric try-on from a single image. Extensive experiments show that our method achieves high-quality, controllable fabric transfer while preserving garment geometry and illumination, without requiring costly reconstruction or rendering pipelines. Our project is available at: https://style3d.github.io/fabric_tryon.