基于改进CNN-Transformer与迁移学习的水泥熟料f-CaO含量预测

Prediction of f-CaO Content in Cement Clinker Based on Improved CNN-Transformer and Transfer Learning

  • 摘要: 针对水泥熟料中游离氧化钙(f-CaO)检测滞后、现有数据驱动模型在非线性小样本条件下精度与泛化性不足的问题,提出了一种融合改进CNN-Transformer与迁移学习的小样本f-CaO预测方法。首先,采用滑动平均法和相关分析筛选关键工艺变量,并构建具有增强特征提取能力的CNN-Transformer混合模型。在此基础上,利用源生产线的大样本数据进行预训练,进而设计4种迁移学习策略,通过有选择地冻结或微调卷积层、交叉注意力层、Transformer编码器及输出层参数,使模型适应目标生产线的小样本数据。实验结果显示,改进后的CNN-Transformer模型预测性能显著提升,均方误差(MSE)降低约63%,决定系数(R2)提高至0.95;在小样本场景下,迁移学习有效增强了模型在目标生产线的泛化能力,其中冻结卷积与交叉注意力层并微调Transformer编码器与输出层的策略效果最优。研究表明,所提方法能够显著缓解f-CaO预测中的小样本制约,在跨生产线复杂工况中具有良好的鲁棒性与应用潜力。

     

    Abstract: To address the lag in f-CaO detection and the insufficient accuracy and generalization of existing data-driven models under nonlinear, small-sample conditions, we propose a small-sample f-CaO prediction method integrating an enhanced CNN-Transformer model with transfer learning. First, sliding average method and correlation analysis are employed to screen key process variables, followed by the construction of a hybrid CNN-Transformer model with enhanced feature extraction capabilities. Building upon this foundation, the model undergoes pre-training using large-scale data from the source production line. Subsequently, four transfer learning strategies are designed. By selectively freezing or fine-tuning parameters in the convolutional layers, cross-attention layers, Transformer encoder, and output layer, the model adapts to the small-sample data of the target production line. Experimental results demonstrate that the improved CNN-Transformer model achieves significantly enhanced prediction performance, with mean squared error (MSE) reduced by approximately 63% and coefficient of determination (R2) increased to 0.95. In small-sample scenarios, transfer learning effectively enhances the model's generalization capability on the target production line, with the strategy of freezing convolutional and cross-attention layers while fine-tuning the Transformer encoder and output layer yielding optimal results. The study demonstrates that the proposed method significantly alleviates the small-sample constraints in f-CaO prediction, exhibiting robust performance and application potential across complex production line conditions.

     

/

返回文章
返回