MMIM: An Interpretable Regularization Method for Neural Networks (Student Abstract)

In deep learning models, most of network architectures are designed artificially and empirically. Although adding new structures such as convolution kernels is widely used, there are few methods to design new structures and mathematical tools to evaluate feature representation capabilities of new structures. Inspired by ensemble learning, we propose an interpretable regularization method named Minimize Mutual Information Method(MMIM), which minimize the generalization error by minimizing the mutual information of hidden neurons and provides ideas for designing new structures. The experimental results also verify the effectiveness of our proposed MMIM.