Abstract: This study proposes a general control system for underwater manipulation, which combines deep reinforcement learning and domain randomization for autonomous underwater manipulation of underwater manipulators. First, a reinforcement learning-based robot control system is established. Subsequently, multi-parameter domain randomization is used to improve the policy robustness and transferring effectiveness, including parameters of manipulator dynamics, hydrodynamic parameters, and noise and delay of state and action spaces. Finally, the trained policy is deployed on a new simulation environment and real underwater arm. The experimental results verify the validity of the proposed method and lay a foundation for autonomous manipulation in the real deep-sea environment in the future.