In this study, a ovel deep learning model named ShuffleVT was proposed to estimate continuous finger force base on surface electromyography (sEMG). This model was composed of the basic units of ShuffleNetV2 and the Vision Transformer (ViT) structure. Its performance was validated on a publicly available NinaPro database, which includes sEMG data from 40 healthy subjects and finger force data from 6 degrees of freedom. Three performance metrics including Pearson correlation coefficient (CC), root mean square error (RMSE), and coefficient of determination (R2) were used. And another four deep learning models, ShuffleNetV2, ViT, Transformer, and LSTM were included for comparison. The results showed the averaged CC, RMSE, and R2 was 0.92±0.05, 1.27±0.66, and 0.83±0.10, respectively, significantly better than that computed with another four models. It indicates that the newly proposed ShuffleVT model could by potentially applied into the sEMG-driven continuous estimation of human motor intention.