Multi-action Cooperative Grasping Strategy for Robotic Arms in Complex Scenes

In complex, uncertain real-world environments, challenges such as disorder, occlusion, and self-occlusion of grasped objects hinder robots from effectively perceiving scenes and executing precise grasps. To tackle these issues, researchers have proposed an active visual framework to enhance scene perception strategies. By coordinating viewpoint adjustments with grasping tasks, this approach aims to improve scene information acquisition and object separation. However, most current methods employ top-down actions, limiting the robot's scene perception capabilities. This study introduces an active visual perception and grasping coordination strategy in a 6-degree-of-freedom (6DoF) pose space. Utilizing deep reinforcement learning, we develop a viewpoint adjustment network, a 4-degree-of-freedom (4DoF) grasping network, and a 6DoF grasping network to learn optimal collaborative strategies. Actions are determined using Q-functions and constraints to execute suitable primitive actions. To enhance scene perception, we propose a scene fusion method following viewpoint adjustment, which integrates information from multiple viewpoints into fixed-size height maps. Experiment results demonstrate an 8.93% increase in captured scene area compared to top-down methods in single viewpoint scenarios, providing comprehensive information for grasping tasks. In cluttered scenes containing ten target objects, the grasping success rate reaches 89.53%. Compared to the state-of-the-art VPG algorithm, our proposed method achieves a 12.02% increase in grasping success rate.