Driver distraction behaviour detection is of crucial significance for the development of driver-centered human-vehicle co-driving systems. Aiming at the existing convolutional neural network-based driver distraction detection models that lack global feature extraction capability, have weak generalisation performance and neglect of the importance of different regions in the driving scene, a driver distraction detection model based on deep learning is constructed to achieve accurate prediction of driver distraction behaviour. First, a residual structure based on HorNet is developed to enhance the feature representation capability through higher-order spatial interactions; second, inspired by the human attention mechanism and the existing attention mechanisms, an adaptive weighted attention strategy is designed to extract the features most relevant to the driving behaviour; and then, the model in this paper is trained on the existing categorical dataset, and the a priori knowledge is used as the initial weights to improve the training results which in turn improves the generalisation ability of the model; finally, the driving behaviour features are visualised to improve the trust in this paper′s model. The experimental results show that the model in this paper can accurately detect driver distraction behaviour, which is significantly better than existing methods in terms of accuracy, and reliability.