Automatic Temporal Relation in Multi-Task Learning

Multi-task learning with temporal relation is a common prediction method for modelling the evolution of a wide range of systems. Considering the inherent relations between multiple time points, many works apply multi-task learning to jointly analyse all time points, with each time point corresponding to a prediction task. The most difficult challenge is determining how to fully explore and thus exploit the shared valuable temporal information between tasks to improve the generalization performance and robustness of the model. Existing works are classified as temporal smoothness and mean temporal relations. Both approaches, however, utilize a predefined and symmetric task relation structure that is too rigid and insufficient to adequately capture the intricate temporal relations between tasks. Instead, we propose a novel mechanism named Automatic Temporal Relation (AutoTR) for directly and automatically learning the temporal relation from any given dataset. To solve the biconvex objective function, we adopt the alternating optimization and show that the two related sub-optimization problems are amenable to closed-form computation of the proximal operator. To solve the two problems efficiently, the accelerated proximal gradient method is used, which has the fastest convergence rate of any first-order method. We have preprocessed six public real-life datasets and conducted extensive experiments to fully demonstrate the superiority of AutoTR. The results show that AutoTR outperforms several baseline methods on almost all datasets with different training ratios, in terms of overall model performance and every individual task performance. Furthermore, our findings verify that the temporal relation between tasks is asymmetrical, which has not been considered in previous works. The implementation source can be found at https://github.com/menghui-zhou/AutoTR.