Dynamic Feedforward Neural Network Architecture Design Based on Information Entropy

To solve the problem of feedforward neural network architecture design, a dynamic feedforward neural network architecture design method based on information entropy is presented. In this method, the neural network's cost functions are composed of the cross entropy of the neural network's expected and actual output and Renyi's entropy of the hidden node's output. It does not require the learning samples to obey the Gauss distribution. In the learning process, the number of the hidden neurons is dynamically adjusted by splitting the most active hidden neurons and removing the least active hidden neurons. This approach can improve the neural network's dynamic response ability and solve the problem of self-organizing architecture design of the feedforward neural network. The proposed method is applied to online modeling of ammonia nitrogen in the wastewater treatment process based on actual operating data. The experiment illustrates the dynamic response capability and the online learning capacity of the neural network.