Universal approximation with neural networks on function spaces

Neural networks play a central role in the construction of learning models for artificial intelligence and machine learning. This is because neural networks are highly flexible and can approximate a wide variety of maps with high accuracy. The flexibility of neural networks is theoretically guaranteed by using universal approximation theorems. When the input and output spaces have finite dimensions, the universal approximation property of neural networks has been intensively investigated under various conditions of width, depth, and activation functions. However, these finite-dimensional results cannot be directly applied to settings in which an input or output space has infinite dimensions, such as functional data analysis and neural processes. This study provides a universal approximation theorem with neural networks for uniformly continuous maps between function spaces, whose dimensions can be infinite.