In order to improve the accuracy of tomato disease recognition based on digital image, to meet the requirements under different resolution when application, and satisfy the uncertainty of actual shooting conditions, a recognition model of tomato leaf disease was built based on image fusion feature. By using the tomato late blight, Mosaic, and early blight leaf image as the research object, four H-components of HSV color model were selected as color features, and the mean, contrast and entropy of gray difference statistics were used as texture features. The seven dimension feature vector is used as the input of SVM classifier, and particle swarm optimization algorithm is used to optimize the SVM model parameters. The test results show that the accuracy of the model is up to 90%, which recognised tomato leaf disease based on image fusion feature.