Aiming at the issue of low accuracy of existing visual defect detection algorithms for magnetic tile appearance recognition, an algorithm based on S-ASPP and dual attention mechanism is proposed. Firstly, in order to solve the problems of low detection efficiency and high hardware cost for algorithm deployment caused by large amount of model parameters and complex network structure, a lightweight GhostNet backbone network is proposed to extract low-level visual features. Secondly, the S-ASPP module is designed to introduce dense connections and depthwise separable convolutions, so as to reduce the amount of model parameters and improve the prediction speed of the model. Next, in order to solve the problem that semantic information may be lost during feature extraction, a dual-attention module is designed. The middle-level features of GhostNet are feed into the dual-attention module and concatinate with the high-level features after ASPP processing, so that the network can acquire more semantic features and improve the segmentation accuracy. Finally, in order to verify the effectiveness of the proposed method, the magnetic tile defect detection algorithm is compared with DeepLabV3+, PSPNet and U-NET algorithms on the magnetic tile data set. Experimental results show that the magnetic tile defect detection algorithm based on separable ASPP and dual attention mechanism has high recognition accuracy, and the average crossover ratio reaches 82.43%. The average pixel accuracy of the category reached 93.08%. The algorithm balances the relationship between the recognition accuracy and the number of parameters and has the best comprehensive performance.