Aiming at the problems that most of the current citrus target detection models need to run on the server, it is difficult to deploy directly in the orchard and the recognition real-time is poor, a portable citrus fruit recognition system based on edge computing equipment is designed. The system consists of an optimized target detection model and an embedded intelligent platform. By extending the YOLOv4-Tiny target detection algorithm, all batch normalization layers are merged into the convolution layer to speed up the forward reasoning speed of the model. Multi-scale structure and K-means clustering method were used to obtain the prior frame size of citrus data set, which made the network model more robust to citrus fruit recognition. GIOU distance measurement loss function was used to measure the loss function, which make the network model pay more attention to the overlapping occlusion area in citrus images. The improved algorithm is deployed to embedded Jetson Nano platform to realize edge detection. The results showed that the average accuracy of the recognition system was 93.01%. The inference time of a single image is about 150 ms, and the video recognition rate is 16 frames/s.