In order to identify the structural damage status with a small amount of unlabeled monitoring data, an innovative damage identification method is proposed in this paper. Firstly, the dynamic response data measured by each long-gauge fiber Bragg grating sensor in the experiment is subjected to variational modal decomposition, and the time-domain and frequency-domain characteristics of each component signal are extracted. The sensitive features of each sensor signal are selected by Relief-F, combined into a sensitive feature set and input into the affinity propagation clustering for damage identification. Two different sets of experiments are used to verify the effectiveness and robustness of the method, and damage recognition rates of 100% and 98.7% are obtained in the two sets of experiments. The results show that the method has practical application value.