Abstract: With the rapid development of MEMS, study of micro helicopter has been a hotpot in this field. Because of its overall size, the micro helicopter could not be equipped with strong sensors and MPU, which affect the helicopter to get the whole environment information, therefore, the traditional control method disagrees with the helicopter in the uncertain environment. However, the method based on behavior only uses trial and error without the exact model of the environment. We adopt reinforcement learning with replacing eligibility traces to be combined with the temporal difference learning, which improves the efficiency and speed convergence. The results of simulation prove the validity of the learning algorithms. At last, this paper introduces the existent problems with the helicopter control and gives the future study trend.