Aiming at the problem that the tracking algorithm based on twin network is prone to degradation under the condition of fast moving target, large deformation and complex background, a target tracking algorithm based on the integration of attention mechanism and adaptive template updating is proposed. Based on SiamRPN, the tracking algorithm combines the channel attention mechanism and the spatial attention mechanism in the feature extraction network to suppress the interference information in the image, supplement the target feature information in the channel space, and better locate the target. The template of the object at different times, including the initial template, the accumulated template and the predicted template, is taken as the input of the residual module. The residual learning strategy is adopted to make full use of the semantic information of the initial template and adaptively update the template needed for the current frame, which reduces the phenomenon of tracking drift. Experimental results on the OTB100 dataset show that the proposed tracking algorithm achieves higher tracking success rate and accuracy compared with other tracking algorithms.