Correntropy-Based Constructive One Hidden Layer Neural Network




Nayyeri, Mojtaba; Rouhani, Modjtaba; Yazdi, Hadi S.; Mäkelä, Marko M.; Maskooki, Alaleh; Nikulin, Yury

PublisherMPDI

2024

Algorithms

Algorithms

49

17

1

1999-4893

1999-4893

DOIhttps://doi.org/10.3390/a17010049

https://www.mdpi.com/1999-4893/17/1/49

https://research.utu.fi/converis/portal/detail/Publication/386866764



One of the main disadvantages of the traditional mean square error (MSE)-based constructive networks is their poor performance in the presence of non-Gaussian noises. In this paper, we propose a new incremental constructive network based on the correntropy objective function (correntropy-based constructive neural network (C2N2)), which is robust to non-Gaussian noises. In the proposed learning method, input and output side optimizations are separated. It is proved theoretically that the new hidden node, which is obtained from the input side optimization problem, is not orthogonal to the residual error function. Regarding this fact, it is proved that the correntropy of the residual error converges to its optimum value. During the training process, the weighted linear least square problem is iteratively applied to update the parameters of the newly added node. Experiments on both synthetic and benchmark datasets demonstrate the robustness of the proposed method in comparison with the MSE-based constructive network, the radial basis function (RBF) network. Moreover, the proposed method outperforms other robust learning methods including the cascade correntropy network (CCOEN), Multi-Layer Perceptron based on the Minimum Error Entropy objective function (MLPMEE), Multi-Layer Perceptron based on the correntropy objective function (MLPMCC) and the Robust Least Square Support Vector Machine (RLS-SVM).

Last updated on 2025-11-02 at 09:58