Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic\n Information Preserving

With widening deployments of natural language processing (NLP) in daily life,\ninherited social biases from NLP models have become more severe and\nproblematic. Previous studies have shown that word embeddings trained on\nhuman-generated corpora have strong gender biases that can produce\ndiscriminative results in downstream tasks. Previous debiasing methods focus\nmainly on modeling bias and only implicitly consider semantic information while\ncompletely overlooking the complex underlying causal structure among bias and\nsemantic components. To address these issues, we propose a novel methodology\nthat leverages a causal inference framework to effectively remove gender bias.\nThe proposed method allows us to construct and analyze the complex causal\nmechanisms facilitating gender information flow while retaining oracle semantic\ninformation within word embeddings. Our comprehensive experiments show that the\nproposed method achieves state-of-the-art results in gender-debiasing tasks. In\naddition, our methods yield better performance in word similarity evaluation\nand various extrinsic downstream NLP tasks.\n