A Weakly Supervised Fine Label Classifier Enhanced by Coarse Supervision

Objects are usually organized in a hierarchical structure in which each coarse category (e.g., big cat) corresponds to a superclass of several fine categories (e.g., cheetah, leopard). The objects grouped within the same coarse category, but in different fine categories, usually share a set of global features; however, these objects have distinctive local properties that characterize them at a fine level. This paper addresses the challenge of fine image classification in a weakly supervised fashion, whereby a subset of images is tagged by fine labels (i.e., fine images), while the remaining are tagged by coarse labels (i.e., coarse images). We propose a new deep model that leverages coarse images to improve the classification performance of fine images within the coarse category. Our model is an end-to-end framework consisting of a Convolutional Neural Network (CNN) which uses fine and coarse images to tune its parameters. The CNN outputs are then fanned out into two separate branches such that the first branch uses a supervised low rank self-expressive layer to project the CNN outputs to the low rank subspaces to capture the global structures for the coarse classification, while the other branch uses a supervised sparse selfexpressive layer to project them to the sparse subspaces to capture the local structures for the fine classification. Our deep model uses coarse images in conjunction with fine images to jointly explore the low rank and sparse subspaces by sharing the network parameters during the training which causes the data obtained by the CNN to be well-projected to both sparse and low rank subspaces for classification.