基于Winograd卷积的并行深度卷积神经网络优化算法

Winograd-based Parallel Deep Convolutional Neural Network Optimization Algorithm

  • 摘要: 针对并行深度卷积神经网络算法在大数据环境下存在冗余特征计算过多、卷积运算性能不足和参数并行化合并效率低等问题,提出了基于Winograd卷积的并行深度卷积神经网络优化算法。首先,该算法提出基于余弦相似度与归一化互信息的特征过滤策略,通过先筛选后融合的方式消除了通道间对于冗余特征的计算,以此解决了冗余特征计算过多的问题;然后,结合MapReduce提出了并行Winograd卷积策略,通过使用并行化Winograd卷积运算替换传统卷积运算的方式来提升卷积运算的性能,以此解决了卷积运算性能不足的问题;最后,提出基于任务迁移的负载均衡策略,通过动态负载迁移的方式来均衡集群中各节点之间的负载,降低了集群总体的平均反应时长,以此解决了参数并行化合并效率低的问题。实验表明,WP-DCNN算法显著降低了DCNN在大数据环境下的训练代价,而且对并行DCNN的训练效率也有大幅提升。

     

    Abstract: To solve the problems of excessive computation of redundant features, insufficient convolution performance, and low efficiency of parallel combining of parameters in the parallel deep convolutional neural network algorithm in the big data environment, we propose the Winograd-based parallel deep convolutional neural network optimization algorithm (WP-DCNN). First, we design a feature filtering strategy based on cosine similarity and normalized mutual information, which solves the problem of excessive computation of redundant features by eliminating the calculation of redundant features between channels through filtering and fusion. Second, we present a MapReduce-based Parallel Winograd Convolution strategy, which solves the problem of insufficient convolution performance by replacing the traditional convolution with the parallelized Winograd convolution to improve the convolution performance. Finally, we present a load-balancing strategy based on task migration, which solves the problem of low efficiency of parallel combining parameters by dynamically migrating loads to balance the load among the cluster nodes and reduce the average response time of the cluster. Experiments show that the proposed algorithm significantly reduces the training cost of DCNN in big data environments and improves the training efficiency of parallel DCNN.

     

/

返回文章
返回