AlignVid: Taming Visual Dominance via Training-Free Attention Modulation in Text-guided Image-to-Video Generation

Text-guided image-to-video generation has made substantial progress, yet it still struggles to execute text-specified edits that require substantial changes to a reference image (e.g., object addition, removal, or modification). Empirically, our analysis reveals that this stems from visual dominance, where the reference image causes severe attention dispersion, inhibiting the model's ability to incorporate new semantic information. To address this, we propose AlignVid, a training-free intervention that recalibrates the model's internal attention distribution. Drawing on an energy-based perspective of attention, AlignVid employs Attention Scaling Modulation (ASM) to reduce attention entropy and concentrate focus on semantic tokens, alongside Guidance Scheduling (GS) to maintain generation stability. To rigorously assess this capability, we present OmitI2V, a comprehensive benchmark for evaluating prompt adherence across object modification, addition, and deletion. Extensive experiments demonstrate that Align-Vid effectively enhances semantic fidelity with negligible computational overhead. Code and the OmitI2V benchmark are available at https: //github.com/LAW1223/AlignVid.

Image-path: OmitI2V/deletion/vanish/nature/2.jpg Prompt: The lush green mountain gradually erodes and disappears, leaving behind rolling sand dunes and a barren desert landscape.

Expected change: The green vegetation and rocky outcrops of the mountain fade away until only dunes of sand remain.