Abstract: We propose a batch process control scheme without a process model by combining reinforcement learning (RL) to solve the problem that conventional model-based control methods have inaccurate models because of their complex nonlinear dynamics when dealing with batch process tasks, which affects control performance. First, the method solves the problem of high estimation of the value function in deep RL algorithms by the structure of twin-actor parallel training to improve the learning efficiency of the algorithm. Second, an independent experience pool is established for each actor to maintain the independence of the twin actors. Furthermore, a novel reward function is established for the RL controller to guide the process back to the predetermined trajectory; we mitigate the temporal difference (TD) error accumulation problem in parameter updating by introducing a delayed policy update method. Finally, the effectiveness of the controller based on the twin-actor deep deterministic policy gradient algorithm for batch process control is demonstrated by simulating the penicillin fermentation process.