Conditional Balance: Improving Multi-Conditioning Trade-Offs in Image Generation

Microsoft Corporation https://nadavc220.github.io/conditional-balance.github.io/ Figure 1. Balanced Conditioning Image Generation. We analyze the sensitivity of model layers to various aspects in conditional inputs. This allows to limit the inputs only to specific layers during inference, thereby balancing the different conditions and preventing content and style from overshadowing each other. As a result, the generative model reduces artifacts and gains artistic freedom when combining complex conditional inputs. As can be seen, by selecting only highly sensitive layers of style (λS) and structure (λT ) we get better color and texture control and better geometric style control.