In order to address the problem that the traditional calibration method will face degraded calibration performance or even complete failure in non-uniform background fields, this paper designs a new calibration method for vector magnetometer arrays in non-uniform background fields, establishes a new error correction model based on the rotationally invariant property of the tensor invariant at a point in space, and uses a numerical method to estimate the calibration parameters of the system. After simulations and field tests, the scaling of the magnetic gradient tensor matrix corrected by this tensor-invariant-based correction (TI) method is approximated to be a constant under various background fields, which is consistent with the theoretical analysis, indicating that the TI correction algorithm has satisfactory correction performance under different background fields. The results show that this correction method can be applied not only in non-uniform fields but also in uniform field environments, with better real-time performance and better correction effect than the conventional method.