Abstract: The problems in the content-based image retrieval are analyzed, and then the ontology theory method is usedto build the low-level feature ontology of the image and the special image ontology. Meanwhile, the concept of imagedescriptor is defined and corresponding image combination rules are setup. At last, the low-level features are selected toretrieve the images, and the multi-class support vector machine (SVM) is chosen to achieve the conjunction between thelow-level feature and the high-level description information. Then the semantic retrieval is realized and the influence of thesemantic gap problem on the content based image retrieval is reduced. The experiment results show that this model canenhance the precision in content based image retrieval, and it can achieve the semantic retrieval by 3～5 times of feedback.