Bird′s nest encroachment is a frequent fault of transmission lines. Birds nesting on the tower will affect the insulation performance of the tower, resulting in tripping accidents. Traditional bird′s nest identification methods for transmission lines are inefficient and lack of security. Therefore, this paper proposes a bird′s nest detection algorithm for transmission lines based on improved YOLOv5 model. By adding CBAM attention module to the backbone network, the feature extraction ability of the backbone network can be improved with less computational cost. The adaptive feature fusion module is introduced into the neck network to replace the original structure and enhance the multi-scale feature fusion effect. The more stable and smooth Mish activation function is used as the activation function to improve the classification accuracy and generalization ability. Experimental results show that, compared with the original YOLOv5s model, the recall rate and average precision of the improved method are improved by 4.4% and 2.3% respectively. It shows good performance for occlusion targets and near-far targets, which verifies the effectiveness of the improved method.