Rate control algorithm dynamically adjusts the encoding parameters to ensure stable transmission rate within the limited bandwidth, and maximize the quality of the video. The parallel encoding and rate control for HEVC has become advanced research focus. The average bit rate control algorithms under existing parallel structure is restrained by the interdependent, so the coding frame can’t timely obtain the actual bits of frames which are in parallel with it. Therefore, in this paper, it uses the model which can predict frame’s number of bits in parallel architecture to control bitrate, and on this basis the optimization process is proposed of buffer. The results show that the method reduces the rate of error by 4.43%, and improves the average Peak SignaltoNoise Ratio (PSNR) of 0.21 dB simultaneously.