The detection and grading system of small grain coffee bean was designed based on the machine vision. The system consisted of the feeding part, the sizing part, the color selection part, the pneumatic part, the receiving part and the electric control box. The system analysis and control software based on OpenCV and visual studio was developed to realize the detection and classification of coffee bean diameter and baking degree. Based on Blob analysis method, the image was extracted by using the minimum circumscribed rectangle method. The color feature value of small coffee beans was extracted by HSV color space model. The degree of baking is identified. Finally, the small–grain coffee green beans are divided into five grades and the degree of baking is divided into three levels: shallow, medium, and deep. The verification test results of the system showed that the average error of the detection of the diameter of the small bean coffee beans was 1.275% and the accuracy of the baking degree detection was 88.9%.