Reconstructing Animatable Categories from Videos

Internet Videos of a Category Canonical Space Articulations & Deformations Differentiable Rendering t=0s t=2s sphynx cat cheetah Skeleton Color: Skinning weights Morphology Motion Transfer Figure 1. Given videos of a deformable category and a skeleton, we reconstruct an animatable 3D model that factorizes variations across instances (e.g., cheetah's and sphynx's are both cats but with different shape morphology, skeleton dimensions, and texture) from time-specific variations within an instance (e.g., skeleton articulations and elastic shape deformation). Left: Input videos; Middle-left: 3D shape, skeleton, and skinning weights (visualized as surface colors) in the canonical space; Middle-right: Disentangled between-instance and within-instance variations over time. Right: Morphology and motion transferred across the two instances.