Fast and Efficient DNN Deployment via Deep Gaussian Transfer Learning

Deep neural networks (DNNs) have been widely used recently while their hardware deployment optimizations are very time-consuming and the historical deployment knowledge is not utilized efficiently. In this paper, to accelerate the optimization process and find better deployment configurations, we propose a novel transfer learning method based on deep Gaussian processes (DGPs). Firstly, a deep Gaussian process (DGP) model is built on the historical data to learn empirical knowledge. Secondly, to transfer knowledge to a new task, a tuning set is sampled for the new task under the guidance of the DGP model. Then DGP is tuned according to the tuning set via maximum-a-posteriori (MAP) estimation to accommodate for the new task and finally used to guide the deployments of the task. The experiments show that our method achieves the best inference latencies of convolutions while accelerating the optimization process significantly, compared with previous arts. Preliminaries DNN layers can be represented as several for-loops. Typically, convolutional operations can be represented as a for o in range(0, M): for h in range(0, H): for w in range(0, W): for i in range(0, N): for kh in range(0, KH): for kw in range(0, KW):