To address the issues of decreased detection accuracy and difficulty in small object recognition caused by blurred traffic signs in extreme weather conditions such as rain, fog, and snow, this paper proposes a traffic sign detection algorithm based on an improved RT-DETR. First, data augmentation is applied to the TT100K dataset under simulated extreme weather conditions to enhance the model′s ability to recognize traffic signs in these environments. Second, the Ortho attention mechanism is introduced into the backbone network, which uses orthogonal filters to reduce feature redundancy and prioritize essential channel information, thereby improving the model′s detection accuracy for small objects. Additionally, a high-level screening-feature pyramid network (HS-FPN) replaces the cross-scale contextual feature mixer (CCFM) in the original model. HS-FPN filters and merges low-level feature information using high-level features, enhancing the model′s detection accuracy for low-contrast and blurred targets in extreme weather conditions. Experimental results show that the proposed improved algorithm achieves an average detection accuracy of 87.84%, an improvement of 2.37% over the original RT-DETR model, while reducing the parameter count to 18.22 M, an 8.4% reduction compared to the original model. The model demonstrates higher accuracy in recognizing small objects and targets under extreme weather, contributing significantly to passenger safety.