Dynamic obstacle avoidance is the key to the robot's autonomous movement and safe walking, in the face of complex and changeable indoor scenes, the robot needs to be able to detect obstacles in time and dynamically plan a safe walking route. In this paper， RGB-D depth camera and IMU unit was used to establish a robot environment perception system, multi-modal information such as three-dimensional vision and attitude angle were provided to the robot. At first, build an improved target detection model based on YOLOv4, The YOLOv4-M target detection algorithm was proposed to identify and locate obstacles in color images, and the depth map was aligned with the color map in order to calculate the size information of the obstacle and the distance information between the robot and the obstacle; The model of obstacle avoidance was built on modified artificial potential field method with the obstacle information in the environment and the posture and angle information of the robot movement, to solve the problem that the calculation of the total potential field falls into a local minimum solution. The model was designed with dynamic programming of the walking path, and the decision result was send to the robot chassis control unit to realize the autonomous movement of the robot in unfamiliar scenes. Simulation analysis and physical experiments show that this method can realize autonomous obstacle avoidance of robots. The research of this method provides a basis and reference for the robot to realize obstacle recognition and autonomous movement avoidance by relying only on vision and inertial navigation sensors.