Addressing the challenge of achieving high-precision tracking for low-altitude unmanned aerial vehicle (UAV) trajectories in complex environments, this paper proposes a residual-driven adaptive Kalman filter(RD-AKF) method based on a distributed microphone array. Within this framework, acoustic source observations are acquired through two complementary algorithms of different dimensions: a geometry-based triangulation method that provides directional constraints, and a frequency-domain phase compensation beamforming(FD-PCB) algorithm that captures energy spatial distribution characteristics. These physically complementary observations are integrated as joint measurements into a unified Kalman filtering framework. To mitigate the limitations of fixed-parameter models in handling abrupt noise from dynamic targets and to improve tracking accuracy, the method incorporates a residual-driven adaptive mechanism into the Kalman filter. This mechanism computes the prediction residuals of each observation channel in real-time and dynamically adjusts the measurement noise covariance matrix based on their statistical properties, thereby optimizing data weighting for the joint estimation of the acoustic source′s position and velocity. Comparative experiments were conducted in an indoor environment by tracking the same low-altitude UAV flight trajectory, comparing the proposed method against standalone baseline intersection, FD-PCB localization, and standard Kalman filtering with fixed covariance matrices. The results show that the Kalman-fused approach reduces the root mean square error(RMSE) by at least 24.3% compared to individual algorithms. With the residual-driven adaptation, the RD-AKF further reduces the RMSE by 18.5% and the maximum positioning error by 15.6% compared to the standard Kalman filter, achieving significantly improved tracking accuracy and stability while maintaining reasonable computational cost. The proposed method provides a high-precision and robust solution for dynamic acoustic source tracking in complex scenarios.