ESReLU: A Dynamic Activation Function for Enhancing Deep Learning Performance in Recommendations

Deep learning has seen a tremendous increase in the recent past.A variety of deep learning architectures have been proposed for various kind of tasks.Activation functions (AF) play a critical role in a deep learning model with the ability to learn abstract properties through nonlinear transformations.This study proposes an adaptive AF, ExtendedSigmoidReLU (ESReLU), to improve model's performance.The proposed AF combines the strengths of Sigmoid and ReLU AFs, dynamically adjusting to varying inputs using a tunable parameter, that allows ESReLU to transition between traditional sigmoid and ReLU functions seamlessly.To ensure continuous and smooth behaviour during training, sigmoid introduces strong gradient flow, whereas ReLU introduces nonlinearity and overcomes sparsity.Experiments conducted on various datasets using a collaborative filtering model demonstrate that ESReLU outperforms traditional functions, achieving faster convergence and performance improvements of up to ~3% on accuracy.These results indicate that ESReLU has significant potential to enhance recommendation performance, especially in data-intensive applications.