Incremental Tabular Learning on Heterogeneous Feature Space

Recently, incremental learning has attracted a lot of interest in both research communities and industries. Generally, given a series of data sets sequentially, it tries to achieve good performance on the new data set while maintaining not bad performance on the old ones. Despite the recent success of incremental learning, existing works mainly assume that the coming data set is from the feature space of old ones, i.e., homogeneous feature space. And they adopt one feature extractor to forcibly project different feature spaces into one space. However, this assumption is hard to hold in real-world scenarios. Especially, the attributes of tables may sequentially increase in tabular learning. Thus, classic incremental learning models may hinder their effectiveness. In this paper, we propose a new method, incremental tabular learning on heterogeneous feature space (ILEAHE) to solve this issue. We first propose the ideas that feature extractors should be decomposed into shared and specific extractors to process the shared and specific features across different data sets respectively. Then, we propose a novel measurement named discriminative ability to measure specific extractors. Thus, two kinds of extractors can be discriminated and the specific extractor will more focus on those domain-specific features. We further demonstrate the effectiveness of ILEAHE through empirical studies.