Although great progress has been made in image style transfer based on deep learning, these methods were not took into account the distortion of lines in the generated image. Therefore, new method combining histogram loss with transformed Gramian matrix is proposed. The use of histogram loss in image style transfer is not only enhanced the image, but also made the generated image more stable. Transformed Gramian matrix is similar to Gram matrix, but the former is extracted more complete texture information, and also took into account the spatial arrangement of an image information. The experimental results show that combination of two methods can not only make the generated image without line distortion, but also reduce the number of iterations in image generation.