Do "Undocumented Workers" == "Illegal Aliens"? Differentiating Denotation and Connotation in Vector Spaces

In politics, neologisms are frequently invented for partisan objectives. For example, "undocumented workers" and "illegal aliens" refer to the same group of people (i.e., they have the same denotation), but they carry clearly different connotations. Examples like these have traditionally posed a challenge to referencebased semantic theories and led to increasing acceptance of alternative theories (e.g., Two-Factor Semantics) among philosophers and cognitive scientists. In NLP, however, popular pretrained models encode both denotation and connotation as one entangled representation. In this study, we propose an adversarial neural network that decomposes a pretrained representation as independent denotation and connotation representations. For intrinsic interpretability, we show that words with the same denotation but different connotations (e.g., "immigrants" vs. "aliens", "estate tax" vs. "death tax") move closer to each other in denotation space while moving further apart in connotation space. For extrinsic application, we train an information retrieval system with our disentangled representations and show that the denotation vectors improve the viewpoint diversity of document rankings. Name Corpus Vocab. Num. Sent. Denotation Grounding Connotation Grounding CR BILL Congr. Record 21,170 381,847 legislation title (1,029-class) speaker party (2-class) CR TOPIC Congr. Record 21,170 381,847 policy topic (41-class) speaker party (2-class) CR PROXY Congr. Record 111,215 5,686,864 none (LM proxy) speaker party (2-class) PN PROXY Partisan News 138,439 3,209,933 none (LM proxy) publisher partisan leaning (3-class)