Addressing Mark Imbalance in Integration-free Marked Temporal Point Processes

Marked Temporal Point Process (MTPP) has been well studied to model the event distribution in marked event streams, which can be used to predict the mark and arrival time of the next event. However, existing studies overlook that the distribution of event marks is highly imbalanced in many real-world applications, with some marks being frequent but others rare. The imbalance poses a significant challenge to the performance of the next event prediction, especially for events of rare marks. To address this issue, we propose a thresholding method, which learns thresholds to tune the mark probability normalized by the mark's prior probability to optimize mark prediction, rather than predicting the mark directly based on the mark probability as in existing studies. In conjunction with this method, we predict the mark first and then the time. In particular, we develop a novel neural MTPP model to support effective time sampling and estimation of mark probability without computationally expensive numerical improper integration. Extensive experiments on real-world datasets demonstrate the superior performance of our solution against various baselines for the next event mark and time prediction. The code is available at https://github.com/undes1red/IFNMTPP. [32,16,27]. Recently, we have witnessed a rapid growth of neural MTPP, which models p * (m, t) using neural networks [24,28,41,26,43], due to the capability of learning complicated temporal patterns and computational efficiency [34].

However, existing studies overlook that the distribution of event marks is highly imbalanced in many real-world applications, with some marks being frequent but others rare, as shown in Figure 1 (a). Similar to other machine learning tasks such as classification, the imbalance poses a significant challenge to the performance of the next event prediction, especially for events of rare marks, which are often more important than other marks (e.g., the occurrence of a 7-magnitude earthquake or a retweet from celebrities). By mitigating the impact of mark imbalance, this study aims to improve the performance of MTPP for next event prediction.

Various techniques have been investigated to improve the prediction performance of rare classes in classifiers, including resampling the training set, cost-sensitive approaches, and thresholding [17,1,39]. Training data resampling requires a proper resampling ratio. Cost-sensitive approaches require domain knowledge on the importance of different marks in setting the cost [17]. To have a solution with minimum external knowledge and assumptions, this study adopts thresholding, which learns thresholds to tune the mark probability normalized by the prior probability of marks.

Addressing mark imbalance for MTPP using thresholding is not straightforward. In addition to mark prediction, MTPP also needs to predict the time simultaneously. In most existing MTPP studies, the strategy is to predict the time based on p * (t), the probability that the next event time is t, and then predict the mark based on p * (m|t), the probability that the next event mark is m at the predicted time t. Our analysis and experiments show that this strategy is unsuitable for addressing mark imbalance with thresholding. If time changes , the mark probability conditioned on time typically changes and thus requires different tuning thresholds. However, it is implausible to learn the tuning thresholds at all times. So, we propose a strategy that first predicts the mark based on p * (m), the probability that the next event mark is m, and then predicts the time based on p * (t|m), the probability that the next event time is t on the condition that the predicted mark m is the next event mark. Since the mark probability p * (m) is independent of time, applying thresholding to handle mark imbalance is easy.

However, our strategy has its challenges. First, two different improper integrations are required for modeling p * (m) and time prediction, respectively. Second, sampling p * (t|m) to predict time is inefficient because it needs the Cumulative Distribution Function (CDF) of p * (t|m), but the CDF does not have a closed-form expression. To overcome these challenges, we find a way to unify the two improper integrations into one. Then, we develop a novel MTPP model, called Integration-free Neural Marked Temporal Point Process (IFNMTPP), to approximate the unified improper integration, rather than using a computationally expensive numerical method. With IFNMTPP, we can directly model p * (m) and the CDF of p * (t|m). The CDF makes drawing samples from p * (t|m) efficient for time prediction. Based on p * (m), the thresholding method can be applied to address the mark imbalance. Extensive experiments on real-world datasets demonstrate the superior performance of our solution against various baselines for the next event mark and time prediction. The contributions of this study are threefold:

• This study investigates the impact of mark