Abstract: Based on the analysis of the shortcomings and deficiencies of classical approximation algorithms, a new approximation scheme is proposed for a fractional-order differential operator - the optimal rational approximation algorithm. We derive the formula of the optimal rational approximation algorithm, and then obtain the optimal trajectory of the band gain by numerical optimization. The proposed algorithm not only avoids the problems caused by the modified Oustaloup algorithm which required that the frequency band of interest should be symmetrical, but also achieves higher precision. The advantages of the proposed algorithm are validated by numerical examples in both the frequency and time domains.