Efficient Scale-Invariant Generator with Column-Row Entangled Pixel Synthesis
Thuan Hoang Nguyen, Thanh Van Le, Anh Tran
Abstract
Any-scale image synthesis offers an efficient and scalable solution to synthesize photo-realistic images at any scale, even going beyond 2K resolution. However, existing GAN-based solutions depend excessively on convolutions and a hierarchical architecture, which introduce inconsistency and the "texture sticking" issue when scaling the output resolution. From another perspective, INRbased generators are scale-equivariant by design, but their huge memory footprint and slow inference hinder these networks from being adopted in large-scale or real-time systems. In this work, we propose Column-Row Entangled Pixel Synthesis (CREPS), a new generative model that is both efficient and scale-equivariant without using any spatial convolutions or coarse-to-fine design. To save memory footprint and make the system scalable, we employ a novel bi-line representation that decomposes layer-wise feature maps into separate "thick" column and row encodings. Experiments on various datasets, including FFHQ, LSUN-Church, MetFaces, and Flickr-Scenery, confirm CREPS' ability to synthesize scale-consistent and alias-free images at any arbitrary resolution with proper training and inference speed. Code is available at https://github. com/VinAIResearch/CREPS .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on18
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 citations
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional DomainsMatthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil et al.NeurIPS 2020 · 4,036 citations
- Implicit Neural Representations with Periodic Activation FunctionsVincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell et al.NeurIPS 2020 · 4,008 citations
- Training Generative Adversarial Networks with Limited DataTero Karras, Miika Aittala, Janne Hellsten, Samuli Laine et al.NeurIPS 2020 · 2,345 citations
Related papers
- Arbitrary-Scale Image SynthesisEvangelos Ntavelis, Mohamad Shahbazi, Iason Kastanis, Radu Timofte et al.CVPR 2022 · 17 citations
- InfinityGAN: Towards Infinite-Pixel Image SynthesisChieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov et al.ICLR 2022 · 84 citations
- Adversarial Generation of Continuous ImagesIvan Skorokhodov, Savva Ignatyev, Mohamed ElhoseinyCVPR 2021
- Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion TransformersKatherine Crowson, Stefan Andreas Baumann, Alex Birch, Tanishq Mathew Abraham et al.ICML 2024 · 98 citations
- Exploring DCN-like architecture for fast image generation with arbitrary resolutionShuai Wang, Zexian Li, Tianhui Song, Xubin Li et al.NeurIPS 2024 · 16 citations
