The development trend of ' reducing people and improving efficiency ' in coal mine production makes it more and more important to ensure the safety of workers. Aiming at the problems of large amount of data and weak robustness of current miner abnormal behavior detection methods, a miner abnormal behavior recognition method with structured discrete attitude perception is proposed. The Kalman filter technology is used to optimize the behavior perception information obtained based on the nine-axis attitude sensor. After the behavior information is intercepted by the sampling window, the three-channel RGB behavior image is structured according to the axial direction of the attitude perception. Combined with the CTFRN model designed to extract the temporal and spatial characteristics, the temporal and spatial characteristics of the five kinds of miners’ behaviors are accurately extracted, and the miners’ abnormal behaviors are monitored with low computational complexity and high robustness. Compared with other models, the results show that the proposed method has higher accuracy, up to 99.3%. The designed system and recognition method can be used for real-time monitoring of abnormal behavior of miners in actual environment to ensure the safety of miners.