Set-Based Face Recognition Beyond Disentanglement: Burstiness Suppression With Variance Vocabulary

Set-based face recognition (SFR) aims to recognize the face sets in the unconstrained scenario, where the appearance of same identity may change dramatically with extreme variances (e.g., illumination, pose, expression). We argue that the two crucial issues in SFR, the face quality and burstiness, are both identity-irrelevant and variance-relevant. The quality and burstiness assessment are interfered with by the entanglement of identity, and the face recognition is interfered with by the entanglement of variance. Thus we propose to separate the identity features with the variance features in a light-weighted set-based disentanglement framework.

Beyond disentanglement, the variance features are fully utilized to indicate face quality and burstiness in a set, rather than being discarded after training. To suppress face burstiness in the sets, we propose a vocabulary-based burst suppression (VBS) method which quantizes faces with a reference vocabulary. With interword and intra-word normalization operations on the assignment scores, the face burtisness degrees are appropriately estimated. The extensive illustrations and experiments demonstrate the effect of the disentanglement framework with VBS, which gets new stateof-the-art on the SFR benchmarks. The code will be released at https://github.com/Liubinggunzu/set_burstiness.