Addressing the challenges faced by autonomous driving systems in nighttime scenarios, such as low background contrast, image blurring, and complex lighting interference, which lead to poor detection performance, this paper proposes a lightweight RDSM-YOLO model based on the YOLOv8 network. The model focuses on comprehensively enhancing key feature extraction and fusion in low-light scenarios. First, RFAConv is introduced into the backbone and neck networks, utilizing a receptive field attention mechanism to adaptively highlight key spatial features; second, the DynamicConv module is used to reconstruct the C2f module, enabling dynamic aggregation of convolutional kernels to enhance feature expression without increasing FLOPs; simultaneously, the lightweight SPPELAN module replaces the traditional SPPF to fuse multi-scale contextual information; finally, the loss function is upgraded from CIoU to EIoU, explicitly decoupling bounding box geometric elements to accelerate convergence and improve localization accuracy. Experimental results show that RDSM-YOLO achieves a mAP50 of 70% for nighttime vehicle detection on the BBD100k dataset, improving by 1.4% over YOLOv8 while maintaining a model parameter count of only 3.04 M. This paper demonstrates that the proposed model achieves both lightweight design and high accuracy in nighttime vehicle detection, providing a reference for improving nighttime autonomous driving performance.