Although object detection can provide the location, size and category of nearby targets for autonomous vehicles, there are still problems of missed detection and false detection in multi-object detection in dense scenes, so an AD-YOLOv5 vehicle detection model is proposed. Firstly, the C3 module in the feature extraction network is optimized to obtain the C-C3 module using the lightweight structure CBAM attention mechanism, which improves the ability to acquire feature information and reduces the attention to other features; secondly, in the detection head section, the classification and regression tasks are decoupled in order to achieve stronger feature representation; then, the generalized power transform is used to perform the transformation operation on the IoU, and the Alpha-IoU loss function with better robustness is proposed, which improves the detection accuracy of the model and accelerates the convergence speed of the model; finally, to add to the complexity of the sample, the GridMask data enhancement technique was used and experiments were carried out on the processed dataset. The experimental results show that the mean average accuracy of the improved target detection model reaches 72.72%, which is 2.25% higher than the original YOLOv5 model, and the model has a high convergence speed, and the visual comparison experiments intuitively show that the model of this paper can effectively avoid the phenomenon of misdetection and omission detection in dense scenes.