Approximation in $L^p(\mu)$ with deep ReLU neural networks.

We discuss the expressive power of neural networks which use the non-smooth\nReLU activation function $\\varrho(x) = \\max\\{0,x\\}$ by analyzing the\napproximation theoretic properties of such networks. The existing results\nmainly fall into two categories: approximation using ReLU networks with a fixed\ndepth, or using ReLU networks whose depth increases with the approximation\naccuracy. After reviewing these findings, we show that the results concerning\nnetworks with fixed depth--- which up to now only consider approximation in\n$L^p(\\lambda)$ for the Lebesgue measure $\\lambda$--- can be generalized to\napproximation in $L^p(\\mu)$, for any finite Borel measure $\\mu$. In particular,\nthe generalized results apply in the usual setting of statistical learning\ntheory, where one is interested in approximation in $L^2(\\mathbb{P})$, with the\nprobability measure $\\mathbb{P}$ describing the distribution of the data.\n