Aiming at the behavior information implied by human posture in agricultural activity images and the association information implied by human and agricultural tools, a recognition method of agricultural activity behavior based on image feature fusion is proposed. Human posture estimation technology OpenPose is used to extract the joint position information of agricultural behavior, and target detection YOLOv3 is used to extract the position and classification information of agricultural tools in agricultural behavior. These information is used to construct the distance space feature matrix and angle space feature matrix of agricultural behavior, to fuse the above image features. Based on explicit and implicit features, the recognition method EI-SVM was establish to realize the recognition of agricultural activity behavior. The experimental results show that the accuracy of EI-SVM method for agricultural activity behavior recognition is 94.87%, and the accuracy on public data set is 92.39%.