Detecting the wearing of safety helmets by construction workers is an important method to ensure personnel safety. However, existing safety helmet detection methods are mostly manual, which are not only time-consuming and labor-intensive but also inefficient. Moreover, the existing algorithm has low detection accuracy in the face of complex environment or weather. In response to this phenomenon, an improved safety helmet wearing detection algorithm is proposed based on the YOLOv5s algorithm. Firstly, the SLSKA-POOL module is proposed based on the residual idea and large separable module design, and used in the pooling layer. This module can make the network pay more attention to the target features and further improve the network capability; secondly, the CAKConv convolutional module is proposed, which efficiently extracts features through irregular convolution operation to improve the network performance; finally, EMA modules are added to the backbone to aggregate multi-scale spatial structure information and establish short and short dependencies to achieve better performance. The experimental results show that: the improved YOLOv5 compared with the original algorithm, The detection accuracy increased by 2.2%, mAP@0.5 increased by 3.6%, and mAP@ 0.5:0.95 increased by 6.4%, realizing more accurate and efficient helmet wearing detection.