A novel type of Ant Colony Algorithm (ACA) for the globally optimal trajectory planning of Unmanned Aerial Vehicle (UAV) is proposed in this paper. The parallelism and positive feedback of ACA is feasible in UAV trajectory planning under complex environments, but the basic ACA model has the limitation of stagnation, and easy to fall into local optimum. Hybrid improvement strategies for the basic ACA model are proposed in this paper, and a type of trajectory smoothing scheme is also put forward. Simulation results show that the improved ACA is effective and can be used in the real-time trajectory planning of UAV. It has also been verified that the proposed method has better performance in convergence speed, solution variation, dynamic convergence behavior, and computational efficiency than the UAV trajectory planning method based on the basic ACA model under complex environments.