As an important haptic perception dimension of texture, the friction feature has a significant impact on the haptic realism of virtual textures. Previous studies utilize traditional physical friction models to model surface friction. However, such methods are often accompanied by high computational complexity and cumbersome parameter setting. To avoid complex texture modeling processes and predict real-time sliding friction that needs to be fed back to the user when interacting with virtual textures, this study establishes an end-to-end texture friction prediction model (TFPM) based on an encoderdecoder that integrates attention mechanisms. This model takes friction data from the previous period and the user′s action information as inputs, which can generate real-time friction signals with high accuracy. It shows a strong generalization effect when dealing with common textures. Subsequently, a haptic device with the function of real-time collection of operation information (pressing pressure and sliding speed) is developed. By combining with the Touch device, data was collected when interacting with 70 real textures, and it was used in conjunction with the SENS3 database to train the model. In order to further verify the generalization ability of the model, a performance evaluation experiment is carried out for the texture samples in the test set. The results show that the model can render the frictional properties of virtual textures with high quality (root mean square error is 0.025 7), and can effectively model the tactile textures outside the database. Finally, the optimal gain parameters of various virtual texture friction signals are determined through psychophysical experiments. Based on this, three user experience experiments are carried out. The experimental results show that the proposed method achieves the highest perceived average similarity score currently (6.25), which can bring users a more realistic virtual texture interaction experience.