Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs
Hui-Po Wang, Ning Yu, Mario Fritz
Abstract
While Generative Adversarial Networks (GANs) show increasing performance and the level of realism is becoming indistinguishable from natural images, this also comes with high demands on data and computation. We show that state-of-the-art GAN models -such as they are being publicly released by researchers and industry -can be used for a range of applications beyond unconditional image generation. We achieve this by an iterative scheme that also allows gaining control over the image generation process despite the highly non-linear latent spaces of the latest GAN models. We demonstrate that this opens up the possibility to re-use state-of-the-art, difficult to train, pre-trained GANs with a high level of control even if only black-box access is granted. Our work also raises concerns and awareness that the use cases of a published GAN model may well reach beyond the creators' intention, which needs to be taken into account before a full public release. Code is available at https://github.com/a514514772/hijackgan .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext fda101a2-0012-466f-a1b5-fa209775bfb1Cited by top-tier papers8
- Dual Contrastive Loss and Attention for GANsNing Yu, Guilin Liu, Aysegul Dundar, Andrew Tao et al.ICCV 2021 · 69 citations
- StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video GenerationYuhan Wang, Liming Jiang, Chen Change LoyICCV 2023 · 20 citations
- Attribute-specific Control Units in StyleGAN for Fine-grained Image ManipulationRui Wang, Jian Chen, Gang Yu, Li Sun et al.ACM MM 2021 · 13 citations
- ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image EditingBingchuan Li, Tianxiang Ma, Peng Zhang, Miao Hua et al.AAAI 2023 · 11 citations
- PluGeN: Multi-Label Conditional Generation from Pre-trained ModelsMaciej Wolczyk, Magdalena Proszewska, Lukasz Maziarka, Maciej Zieba et al.AAAI 2022 · 8 citations
Builds on13
- Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?Rameen Abdal, Yipeng Qin, Peter WonkaICCV 2019 · 1,195 citations
- GANSpace: Discovering Interpretable GAN ControlsErik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain ParisNeurIPS 2020 · 1,049 citations
- Unsupervised Discovery of Interpretable Directions in the GAN Latent SpaceAndrey Voynov, Artem BabenkoICML 2020 · 459 citations
- On the "steerability" of generative adversarial networksAli Jahanian, Lucy Chai, Phillip IsolaICLR 2020 · 421 citations
- GANalyze: Toward Visual Definitions of Cognitive Image PropertiesLore Goetschalckx, Alex Andonian, Aude Oliva, Phillip IsolaICCV 2019 · 345 citations
Related papers
- Sketch Your Own GANSheng-Yu Wang, David Bau, Jun-Yan ZhuICCV 2021 · 82 citations
- Towards Discovery and Attribution of Open-world GAN Generated ImagesSharath Girish, Saksham Suri, Sai Saketh Rambhatla, Abhinav ShrivastavaICCV 2021 · 87 citations
- Interpreting the Latent Space of GANs for Semantic Face EditingYujun Shen, Jinjin Gu, Xiaoou Tang, Bolei ZhouCVPR 2020
- Designing an encoder for StyleGAN image manipulationOmer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik et al.SIGGRAPH 2021 · 692 citations
- Using latent space regression to analyze and leverage compositionality in GANsLucy Chai, Jonas Wulff, Phillip IsolaICLR 2021 · 30 citations
