In order to solve the long endurance cumulative error and real-time problem of inertial navigation system, a multi-source information fusion algorithm based on factor graph is proposed. Based on sum product algorithm, the factor graph model is used to estimate the state based on the maximum a posteriori probability, the mean and variance of variables are calculated to complete data fusion, and the UWB technology and vision sensor are used to calibrate the inertial navigation system with accumulated error to complete error correction. The simulation results show that the error of each axis is 0.36% and 0.31% respectively, and the solution time is 50ms. The error of this algorithm is only one third of the cumulative error of inertial navigation, and its real-time performance is better than other algorithms.