Distracted driving is one of the main causes of road traffic safety problems. Aiming at the problems of high computational complexity, limited generalization ability and unsatisfactory detection accuracy of existing detection algorithms based on deep learning, this paper constructs a lightweight distracted driving behavior detection algorithm based on improved YOLOv8n. Firstly, the Context Anchor Attention mechanism was fused into StarNet to form StarNet-CAA, and StarNet-CAA was integrated into the backbone network of YOLOv8n to improve the global feature extraction ability of the model and significantly reduce the computational complexity. Subsequently, FasterBlock combined with CGLU is added to the neck network to form the C2f-Faster-CGLU module, which reduces the computational cost. In addition, the shared convolution is introduced into the detection head to further reduce the computational burden and parameter size. Experimental results show that the improved YOLOv8n algorithm significantly improves the efficiency of distracted driving behavior detection, reaching an accuracy of 99.3%on the StateFarm dataset. The number of parameters of the model is reduced by 46.7%, and the amount of calculation is reduced by 41.5%. In addition, the generalization experiment is carried out on the 100-Driver dataset, and the results show that the generalization effect of the proposed scheme is improved compared with YOLOv8n. Therefore, the proposed algorithm significantly reduces the computational burden while maintaining high reliability and generalization ability.