In order to improve the quality of Coix shelling, a pressure-mill-type Coix shelling device was designed based on the principle of flexible shearing and kneading. The device is mainly composed of feeding device, moving mill, pressure mill and pressure adjusting device. During the work process, Coix entered the hulling space with the feeding device, and was rustled and hulled by the pressure and friction force between the two mills. When the pressure mill was larger than the crushing force of Coix, it was jacked up to increase the hulling gap to protect Coix. Through the mechanical analysis of the Coix in the diversion area, the broken shell area and the separation area, the main factors affecting the quality of the Coix shelling were determined to be moisture content, shelling speed, shelling clearance and pre-tightening force. The optimum parameter range of Coix shelling device was determined by single factor test with the threshing rate and damage rate as the test indexes. Based on the single-factor test results, the Coix with moisture content of 7% was selected, and three factors quadratic regression orthogonal rotation combination test was conducted by the hulling device speed, the hulling clearance and the pre-tightening force as the test factors, and the net rate and the damage rate as response index. The test results showed that the main factors affecting the threshing rate of the Coix shelling device were: pre-tightening force, shelling clearance, speed, and the main factors affecting the damage rate of the Coix shelling device were: shelling clearance, pre-tightening force, speed. Based on the response surface method, the regression model was optimized with multiple objectives, and the optimal operation level combination of the shelling device was obtained as follows: speed of 513.625 r/min, shelling clearance of 2.061 mm, and pre-tightening of 119.628 N. At this time, the threshing rate of Coix was 91.731% and the damage rate was 9.612%. The optimized parameters were rounded and continuous hulling was carried out in Coix. The net removal rate was 91.12% and the breakage rate was 8.93%, which could meet the practical production requirements.