Abstract: A autonomous vision-based localization and control method has been developed for unmanned aerial vehicles (UAV) operating in GPS-denied environments. To improve the robustness of the vision-based localization scheme, the visual simultaneous localization and mapping (SLAM) algorithm is enhanced by increasing its number of features and by optimizing its storage of key frames. The map-losing issue is avoided by combining the optical flow algorithm with the original visual SLAM algorithm. By merging the visual SLAM and optical flow algorithms together, short-term precise velocity and long-term drift-free position estimations are achieved simultaneously. To obtain more accurate and faster states estimation, a state-of-the-art EKF algorithm is utilized to fuse the position information within the onboard IMU readings. Based on the fused localization information, a PID and nonlinear robust RISE controller is designed to increase the robustness of the flight controller. The proposed localization and control algorithms are implemented on a self-built quadrotor UAV testbed. To avoid time delays and signal interference from the wireless transmission process, the motion states are estimated through an onboard embedded computer. Outdoor flight experimental results demonstrate that the proposed strategies achieve good autonomous control performance.