Stamatis Mastromichalakis

ALReLU: A different approach on Leaky ReLU activation function to improve Neural Networks Performance

2020 · arXiv (Cornell University)

Despite the unresolved 'dying ReLU problem', the classical ReLU activation function (AF) has been extensively applied in Deep Neural Networks (DNN), in particular Convolutional Neural Networks (CNN), for image classification. The common gradient issues of …