A Novel Correlation Gaussian Process Regression-Based Extreme Learning Machine

Abstract One obvious defect of Extreme Learning Machine (ELM) is that the prediction performance of ELM is sensitive to the random initialization of input-layer weights and hidden-layer biases. GPRELM integrating Gaussian Process Regression (GPR) into ELM is a newly-proposed, simple and effective strategy to make ELM insensitive to the random initialization. However, the kernel based GPRELM (kGPRELM) exists serious over-fitting. In this paper, we analyze the reason for over-fitting of kGPRELM in theory and further propose a correlation based GPRELM (cGPRELM) which uses the correlation coefficient to measure the similarity between two different hidden-layer output vectors. cGPRELM reduces the possibility that covariance matrix becomes an identity matrix with the increase of hidden-layer nodes and thus controls the over-fitting effectively. Meanwhile, cGPRELM works well for improper initialization intervals where ELM and kGPRELM can't provide the good predictions. The experimental results on real classification and regression data sets demonstrate the feasibility and superiority of cGPRELM, which not only obtains the better generalization performance but also has the lower computational complexity.