GreedyNAS: Towards Fast One-Shot NAS With Greedy Supernet

Supernet: a fundamental performance estimator of different architectures (paths). Target Assumption: the supernet should estimate the performance accurately for all paths, and thus all paths are treated equally and trained simultaneously. Issues: 1. It is harsh to evaluate accurately on such a huge-scale search space (e.g. 7 !" ). 2. Training architectures with inferior quality would disturb the weights of those potentially-good paths. 3. Training on those weak paths involves unnecessary update of weights, and slows down the training efficiency.