Let's Chorus: Partner-aware Hybrid Song-Driven 3D Head Animation

Figure 1. Multi-singers Animation. (a): Previous methods construct the 3D facial animation conditioned with an input of single-person audio. (b): With a hybrid song from multi-singers, we argue that it is essential to construct the emotional interaction between each singer for accurate 3D head generation. Motivated by this, we propose the PaChorus framework conditioned on a segment of mixed audio consisting of background music and vocals from multi-singers. With inter-singer interaction modeling, our method can generate emotion-consistent animation sequences.