Abstract: A multi-feature fusion based particle filter algorithm for object tracking is proposed to improve the tracking performance with background confusion and to deal with the nonlinear or non-Gaussian characteristics of state model and observation model in vision object tracking field. An adaptive method of choosing object color histogram is presented to get an accurate color model of the object. To cope with the influence of illumination variation and background confusion, a hybrid algorithm of color histogram and edge orientation histogram under particle filter framework is put forward to improve robustness of object tracking. The results of the experiment show that the proposed algorithm can track both single objects and multi-objects, even with similar color disturbance.