In the process of electromagnetic parameters testing with waveguide transmission line, the measurement reference plane can not be at both ends of the material being measured, resulting in a large measurement deviation.In this paper, the expression of permeability and permittivity of the measured dielectric material is obtained by constructing a test simulation model using transmission reflection measurement method and electromagnetic parameter inversion algorithm.On this basis, in order to solve the measurement reference plane problem, TRL(Through-reflect-line) calibration algorithm is being proposed,as a result，TRL calibration algorithm reduce the deviation of scattering parameters and to 12% and 16% in theory. In order to solve the embedding deviation problem, the de-embedding algorithm is being proposed,and the de-embedding algorithm can reduce the deviation of scattering parameters and to 18% and 6% respectively.The simulation results show that the two optimization algorithms can effectively reduce the measurement deviation and improve the measurement accuracy of electromagnetic parameters.