Aiming at the technical difficulties in the quality inspection process of the textile industry for the defect detection of complexly textured fabrics, an image detection model based on a deep convolutional neural network is proposed. Firstly, the YOLOv7tiny model was selected as the reference frame of the algorithm, and then the optimization was carried out, including using the SimAM module to reconstruct the feature fusion layer so as to improve the model′s ability to extract local features of defects and suppress background features. SIoU was used to optimize the coordinate positioning loss function to speed up the regression efficiency of bounding boxes. The FReLU activation function is introduced to enhance the utilization of spatial information in the nonlinear activation layer and improve the spatial sensitivity of the activation function. The experimental results show that the accuracy and recall ratio of this model are better than those of other existing algorithms in the detection tasks of five typical defects for complex texture fabrics. The mAP reaches the maximum value of 805%, the size of the model is only 92 M, and the detection of a single frame image on the PC is only 2113 ms.