In order to obtain the defect location of the continuous casting billet caused by the failure of the deburring machine, a defect location method based on binocular vision was proposed. Firstly, aiming at the poor matching effect of AD-Census algorithm in casting blank images, a noise detection method of window center pixels is proposed, which replaces noise pixels with neighborhood pixel information, and integrates multi-directional gradient cost into the calculation of cost function to improve the reliability of initial cost; Secondly, an adaptive window cost aggregation based on gradient threshold is designed to improve the matching effect of the algorithm in weak texture region and edge region; Finally, the parallax map of casting billet is converted by three-dimensional coordinate to complete the defect location of continuous casting billet. Experiments show that the binocular vision defect location method proposed in this paper has high parallax map matching accuracy, and the average location error of the casting blank defect depth is less than 1 mm, which can provide reliable location information for the subsequent defect processing device.