Are "Undocumented Workers" the Same as "Illegal Aliens"? Disentangling\n Denotation and Connotation in Vector Spaces

In politics, neologisms are frequently invented for partisan objectives. For\nexample, "undocumented workers" and "illegal aliens" refer to the same group of\npeople (i.e., they have the same denotation), but they carry clearly different\nconnotations. Examples like these have traditionally posed a challenge to\nreference-based semantic theories and led to increasing acceptance of\nalternative theories (e.g., Two-Factor Semantics) among philosophers and\ncognitive scientists. In NLP, however, popular pretrained models encode both\ndenotation and connotation as one entangled representation. In this study, we\npropose an adversarial neural network that decomposes a pretrained\nrepresentation as independent denotation and connotation representations. For\nintrinsic interpretability, we show that words with the same denotation but\ndifferent connotations (e.g., "immigrants" vs. "aliens", "estate tax" vs.\n"death tax") move closer to each other in denotation space while moving further\napart in connotation space. For extrinsic application, we train an information\nretrieval system with our disentangled representations and show that the\ndenotation vectors improve the viewpoint diversity of document rankings.\n