Abstract: The deep reinforcement learning algorithm is widely used in UAV navigation tasks. However, in the training process using the fusion prior strategy, the model training speed is slow, and the success rate of navigation decreases due to the linear attenuation of its proportion. First, we establish a virtual UAV environment model and construct the action space based on UAV autonomous navigation. Next, we design the reward function built on the nonsparsity idea. Coupled with the self-adaptive attenuation factor based on state, the weight of prior policy under the different states is ameliorated. Finally, we realize the autonomous navigation decision-making of UAVs using the trained network model. Simulation results manifest that the training time when the navigation success rate is stable at a high level is reduced by 20% from the prototype algorithm, indicating that we increase the training efficiency and cut down the time cost. In addition, the navigational quality and success rate are slightly enhanced. The proposed algorithm provides a new idea to facilitate the practical use of deep reinforcement learning in UAV autonomous navigation.