Human-Swarm Robots Interaction Approach Based on Virtual Reality and Eye Tracking

Remote control of swarm robots by human operators is fraught with many issues due to the vast number of controlled robots, the difficulty of accessing the state of swarms, and the difficulty of regulating the collective actions of robots.As a result, some conventional interaction methods, such as manual remote control, cannot achieve efficient, flexible, and highly immersive human-swarm robot interactions.We propose a novel human-swarm robot interaction (HSI) approach based on virtual reality (VR) and eye-tracking techniques.First, we construct a VR interaction scene of swarm robots and collect the eye-tracking data of operators.Second, we analytically obtain eye movement features such as pupil and gaze positions and generate the motion control instructions of the swarm robots accordingly.Finally, we test the proposed HSI approach through three experiments using real swarm robots: swarm roaming, swarm goal-oriented tracking, and swarm trajectory tracking.The robot experimental results show that our HSI approach can realize flexible and convenient human-swarm robot remote interactive control, which provides a new form for human-swarm robot interactions.