The Causal-Neural Connection: Expressiveness, Learnability, and\n Inference

One of the central elements of any causal inference is an object called\nstructural causal model (SCM), which represents a collection of mechanisms and\nexogenous sources of random variation of the system under investigation (Pearl,\n2000). An important property of many kinds of neural networks is universal\napproximability: the ability to approximate any function to arbitrary\nprecision. Given this property, one may be tempted to surmise that a collection\nof neural nets is capable of learning any SCM by training on data generated by\nthat SCM. In this paper, we show this is not the case by disentangling the\nnotions of expressivity and learnability. Specifically, we show that the causal\nhierarchy theorem (Thm. 1, Bareinboim et al., 2020), which describes the limits\nof what can be learned from data, still holds for neural models. For instance,\nan arbitrarily complex and expressive neural net is unable to predict the\neffects of interventions given observational data alone. Given this result, we\nintroduce a special type of SCM called a neural causal model (NCM), and\nformalize a new type of inductive bias to encode structural constraints\nnecessary for performing causal inferences. Building on this new class of\nmodels, we focus on solving two canonical tasks found in the literature known\nas causal identification and estimation. Leveraging the neural toolbox, we\ndevelop an algorithm that is both sufficient and necessary to determine whether\na causal effect can be learned from data (i.e., causal identifiability); it\nthen estimates the effect whenever identifiability holds (causal estimation).\nSimulations corroborate the proposed approach.\n