Due to the small pitting area of the ball screw and the serious environmental interference, defects are difficult to detect in time. Therefore, a Multi-level decoding neural network is proposed to realize the segmentation of pitting defects in ball screws. The network consists of an encoder, a multi-level decoder and a Multi-scale Attention module. The encoder is composed of Resnet34, and the Ghost module is introduced to build a lightweight multi-level decoder. In order to fuse multi-scale features and filter redundant information, the Multi-scale Attention module is designed. A hybrid loss function composed of BCE function, IOU and SSIM function is used to train the network. Experiments on the ball screw defect dataset show that Multi-level decoding neural network achieves 0.770 3 in the maxFβ metrics, compared with other methods, which achieves better segmentation results, and the processing time of a single image is 26 ms. It provides a new method for real-time segmentation of ball screw pitting defects.