On the Exploration of Local Significant Differences For Two-Sample Test

Recent years have witnessed increasing attentions on two-sample test with diverse real applications, while this work takes one more step on the exploration of local significant differences for two-sample test. We propose the ME MaBiD , an effective test for two-sample testing, and the basic idea is to exploit local information by multiple Mahalanobis kernels and introduce bi-directional hypothesis for testing. On the exploration of local significant differences, we first partition the embedding space into several rectangle regions via a new splitting criterion, which is relevant to test power and data correlation. We then explore local significant differences based on our bi-directional masked p-value together with the ME MaBiD test. Theoretically, we present the asymptotic distribution and lower bounds of test power for our ME MaBiD test, and control the familywise error rate on the exploration of local significant differences. We finally conduct extensive experiments to validate the effectiveness of our proposed methods on two-sample test and the exploration of local significant differences.

This work presents a new two-sample test from local and directional information, and further explore local significant differences. The main contributions can be summarized as follows:

• We propose the effective ME MaBiD test for two-sample testing, and the basic idea is to exploit local information by multiple Mahalanobis kernels and introduce bi-directional hypothesis for testing. Intuitively, Mahalanobis kernels are more flexible to exploit local differences from neighborhoods and feature maps, and the bi-directional hypothesis is beneficial to improve the sensitivity of two-sample test with proper parameter adaptation.

• We partition the embedding space into several rectangle regions based on a new splitting criterion, which is relevant to test power and data correlation. We introduce the bi-directional masked p-value for each rectangle region, and finally explore local regions with significant difference based on our bi-directional masked p-value together with the ME MaBiD test.

• We present theoretical guarantees for our ME MaBiD test via the asymptotic distribution, as well as the lower bounds on the test power for our test. We also present the upper bounds on familywise error rate for our exploration of local significant differences.

• We conduct extensive experiments to validate the effectiveness and efficiency of our methods. Specifically, our methods achieve better performance on most datasets for two-sample test and exploring local significant differences, along with comparable or smaller running time.

The rest of this work is organized as follows: Section 2 presents our ME MaBiD test. Section 3 explores local significant differences. Section 4 conducts extensive experiments, and Section 5 concludes with future work. All technical proofs are given in Appendix A.