FlexControl: Computation-Aware Conditional Control with Differentiable Router for Text-to-Image Generation

Figure 1. Dynamically inject conditional controls for image generation based on timestep and task-specific sample. (a) The architecture of conventional controllable generative model -ControlNet. (b) The architecture of the proposed FlexControl. (c) Statistics on the number of activated control blocks of the FlexControl at each denoising step. Here, "∼50% spasity" indicates that the number of floating-point operations (FLOPs) of activated blocks is limited to 50% of the trainable branch.