Greg: GEometry-Aware RegIon Refinement for Sign Language Video Generation

Sign Language Video Generation (SLVG) aims to transform sign language sequences into natural and fluent sign language videos. Existing SLVG methods lack geometric modeling of human anatomical structures, leading to anatomically implausible and temporally inconsistent generation. To address these challenges, we propose a novel framework: Geometry-Aware Region Refinement (GReg) for SLVG. GReg uses geometric information (such as normal maps and gradient maps) from the SMPL-X model to ensure anatomical and temporal consistency. To fully leverage the geometric priors, we propose two novel methods: 1) Regional Prior Generation, which uses regional expert networks to generate target-structured regions as generation priors; 2) Gradient-enhanced Refinement, which guides the refinement of detailed structures in key regions using gradient features. Furthermore, we enhance visual realism in key regions through adversarial training on both these regions and their gradient maps. Experimental results demonstrate that GReg achieves state-of-the-art performance with superior structural accuracy and temporal consistency.