At present, glass surface defect detection is mainly manual, which takes a long time and has low accuracy. An improved algorithm model YOLO-M combining YOLOv4 and mobilenetv3 is proposed to solve this problem. Firstly, the mobilenetv3 network is used to replace the original backbone network cspdarknet53 of YOLOv4, and the activation function is modified to improve the running speed on the basis of reducing the model size and parameters. Then, the glass defect samples were photographed and sampled. The defects were divided into wear, bubble and scratch, and the glass defect data set was established. Finally, the glass defect data set is trained by using YOLO-M, YOLOv4 and YOLOv4 tiny algorithms, and the evaluation indexes such as average precision and frame rate under different algorithms are compared. The experimental results show that the frame rate of YOLO-M algorithm in glass defect detection is 57.72 f/s, and the average accuracy is 91.95%. YOLO-M algorithm has obvious effect on the speed and accuracy of glass defect recognition, and can be used as an important reference for subsequent sorting research and other glass product defect recognition.