Abstract: Improved predictions can be obtained by using distributed neural networks(DNNs) instead of an individual better network as usual. New approach to obtain combination weights of multiple networks is derived based on parallel recursive prediction error algorithm. Model accuracy and robustness can be significantly improved by using distributed neural networks. The proposed method has been applied and evaluated for a complex dynamic nonlinear system. Results obtained demonstrate that this approach can improve the performance of neural network based nonlinear models.