This paper proposes a Main-Partial Transfer Region-CNN method for insulator defect detection to address the problem that the Faster R-CNN algorithm is not accurate in detecting insulator defects with small samples in complex environments. The whole method uses a convolutional neural network with a fused residual module and feature pyramid structure as the backbone network for feature extraction, which is used to adapt to different scales of defect targets and retain more valid information. Then, the detected insulator body is automatically cropped to improve the effectiveness of the complex background in the detection of the defective area, so that the model can be more effective in mining the defective features. The insulator defect images collected by UAV aerial photography are detected. The results show that the average accuracy of the method in this paper is improved by 37.5% compared with the Faster R-CNN baseline model, reaching 89.6%. The accuracy is improved by 34.9% and 60.2% on the detection of missing and damaged, respectively.