With the wide application of deep learning technology in rail inspection, visual inspection methods in the field of rail fasteners have been increasingly studied. Aiming at the efficiency bottleneck of constructing defective samples in the current rail fastener data set, and the relative lack of means to detect loose parts based on image data, this paper proposes a rail fastener detection method based on data enhancement and YOLO model. In this study, the line array camera mounted on the inspection vehicle collects images to obtain raw data and texture information, uses the a priori information of the image to control the point cloud data to efficiently generate mask images and label files containing contour information, and realizes the migration and fusion of texture information based on the style migration model. Aiming at the demand of synchronization based on image data to realize the detection of missing and other states and loose states, the attention mechanism and adaptive splicing layer are introduced, and the multi-task detection model is constructed to realize the rapid identification of fastener states and the accurate segmentation of the bolt region, and the average accuracy of target detection reaches 92.14%, and the pixel accuracy of semantic segmentation reaches 89.6%. The method in this paper effectively improves the efficiency of data enhancement and reduces the probability of leakage detection for bolt states in the field of 2D images.