In response to the current industrial field, a SocFPGA architecture solution with high-speed interconnect bus is proposed to address the problem of slow processing speed in surface defect detection in current workpiece inspection. Firstly, to remove defects in the image and highlight the defective pixels, the image data is processed by grayscale, median filtering, adaptive threshold segmentation and morphological filtering. Secondly, the pixel threshold algorithm and voting algorithm were used to label the locations of surface defects in the image. Finally, a surface defect detection system for workpieces was built using copper plate samples and OV5640 cameras. Experiments show that when the defect diameter is not less than 0.5 mm, defect detection rate is about 90.24%, and the processing time per frame is about 0.62 μs. It realizes the real-time online detection of workpiece surface defect image, which provides a certain reference for the research in this field.