Wireless communication technology has rapidly advanced and attracted widespread application. Moreover, the positioning requirements for signal sources in various fields have also increased substantially. The positioning method based on the time difference of arrivals (TDOA) is one of the most widely used passive positioning technology. Recently, machine learning has developed rapidly, leading to new ideas and methods for passive positioning technology. Comparing various passive positioning methods, we first discuss the technical characteristics and advantages of TDOA. Next, we analyze the application and challenges of the positioning algorithm based on the optimization theory, including the time difference estimation method, solution method, non-line-of-sight propagation influence in urban environments, base station selection, geometric distribution, and other aspects. Finally, we review and discuss the latest application of machine learning to optimization theory for improving the performance of passive positioning based on TDOA. We also investigate future development trends and opportunities.