Progressive Neural Architecture Generation

Specifically, MSQ constructs sub-architectures using quantization decoding and progressively expands them, transitioning from simple to complex forms. This operation bypasses network inference to enhance efficiency. Complementing MSQ, SCC, implemented through a tailored regularization mechanism, introduces penalties for deviations during subarchitecture generation, guiding the process towards valid target architectures. As such, PNAG establishes a clear generation path, laying the groundwork for generating suitable architectures in downstream tasks. Extensive experiments demonstrate that PNAG not only generates superior architectures for various downstream tasks (+8.43%/+5.07%, on average) but also significantly improves generation efficiency, reducing the architecture generation time by 1300×. Furthermore, PNAG demonstrates strong extensibility by successfully generating Transformer-based architectures.