To solve the problem of insufficient global searching ability of coati optimization algorithm(COA),this paper proposes an im-proved coati optimization algorithm(ICOA),which initializes the population through SPM chaotic mapping,introduces a comprehensive posi-tion update strategy of Levy flight and lens imaging reverse learning,and improves the ability of jumping out of local optimal solution and glob-al searching.Benchmark function test results show that ICOA has better convergence speed and convergence accuracy than COA,sparrow search algorithm(SSA)and arithmetic search algorithm(AOA).Aiming at the path planning problems of various terrain,threats and con-straints of 3D UAVs,the simulation environment of hilly landform is constructed to simulate the scene of UAVs performing exploration tasks of ancient buildings.Meanwhile,ICOA,COA,SSA and AOA are used for path planning.The simulation results show that the fitness value of the path planned by ICOA algorithm is the best,the path distance is shorter and the pitch Angle is smaller,which further verifies the effectiveness of ICOA and the feasibility of its application in UAV exploration missions in hilly areas.