The edge of electroluminescence image is fuzzy and the texture is not clear, which makes it difficult to quantitatively evaluate the defects of crystalline silicon photovoltaic cell. In order to solve this problem, a defect detection method based on electroluminescence polarization image fusion is proposed. First, based on the analysis of the crystalline silicon photovoltaic cell structure, the electroluminescence polarization imaging mechanism was introduced. Then, the Laplacian pyramid was used to decompose the obtained infrared intensity images and polarization images, and guide filter was used to enhance the high-frequency components. The high and low frequency parts were fused by the rule of regional energy maximum and regional energy weighted average. Finally, a short wave infrared polarization detection platform was established for photovoltaic cell inspection. The results show that polarization imaging can highlight the contour edges and texture detail of the photovoltaic cell defect image. The photovoltaic cell defect features in the fusion image are more prominent. The objective evaluation index such as information entropy and standard deviation are significantly improved, which verifies the effectiveness of the proposed method.