Triple-D: Denoising Distant Supervision for High-Quality Data Creation

Distant supervision is a technique that aims to create large amounts of training data at a low cost. This approach benefits various downstream systems, particularly in natural language processing and relation extraction tasks. However, due to its strong assumption that any sentence containing entities expresses the specific relation between them found in existing knowledge bases (KBs), distant supervision introduces considerable noise. Existing works attempt to denoise distant supervision data by either using the original text or replacing entities in the text with patterns representing the entity types as inputs. However, replacing a frequently repeating pattern will result in loss of context due to the excessively general semantics of the pattern. Furthermore, due to the lack of ground truth, denoising module often relies on parametric models that still learn distribution from noisy data, which further limits model performance. In this paper, we propose Triple-d, a technique for high-quality data creation through adaptive pattern replacement and a scalable non-parametric model. Specifically, we formulate the adaptive pattern replacement task as a maximum-profit bipartite graph problem and propose an approximation algorithm as a solution. Additionally, we design a non-parametric model with scalable instance normalization to efficiently estimate and eliminate the influence of each dimension in neighbors. Extensive experiments in the denoising task and a downstream relation extraction task on real-world datasets demonstrate the superior effectiveness and efficiency of Triple-d, highlighting its potential to improve the performance for high-quality data creation.11Corresponding author: Yongqi Zhang.