Abstract: Based on the model of visual attention, a track damage detection method for high-speed railways is proposed. The visual attention model based on sparse sampling and kernel density estimation is adopted to extract the damaged regions from the salient maps of a high-speed railway video. A semi-supervised classification based on a generative model is proposed to solve the problem of having a small sample in the damaged image classification. Then, it is used in the damaged images' recognition and classification, which makes use of both labeled and unlabeled samples. Three types of typical track damages are examined in the experiment and the results show that the proposed method has a high detection and recognition rate.