上海交通大学学报

上海交通大学学报（自然版） ›› 2012, Vol. 46 ›› Issue (12): 1931-1935.

• 自动化技术、计算机技术 • 上一篇    下一篇

基于Q学习的无人机三维航迹规划算法

郝钏钏a，方舟b，李平a   

  1. （浙江大学a.控制科学与工程学系； b.航空航天学院， 杭州 310027）
  • 收稿日期:2012-05-28 出版日期:2012-12-29 发布日期:2012-12-29
  • 基金资助:

    国家自然科学基金资助项目（61004066），中央高校基本科研业务费专项资金资助项目（2011FZA4031）

A 3-D Route Planning Algorithm for Unmanned Aerial Vehicle Based on Q-Learning

 HAO  Chuan-Chuan-a, FANG  Zhou-b, LI  Ping-a   

  1. (a.Department of Control Science and Engineering; b.School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, China)
  • Received:2012-05-28 Online:2012-12-29 Published:2012-12-29

摘要：  针对现有的基于强化学习的无人机航迹规划方法因无法充分考虑无人机的航迹约束而使规划获得的航迹可用性较差的问题，提出一种更有效的无人机三维航迹规划算法.该算法利用无人机的航迹约束条件指导规划空间离散化，不仅降低了最终的离散规划问题的规模，而且也在一定程度上提高了规划获得的航迹的可用性，通过在回报函数中引入回报成型技术，使算法具有满意的收敛速度.无人机三维航迹规划的典型仿真结果表明了所提出算法的有效性.

关键词: 无人机, 三维航迹规划, 启发信息, 航迹约束, Q学习

Abstract: As the route constraints of the unmanned aerial vehicle (UAV) are neglected in most of the existed route planning algorithms based on reinforcement learning, the resulted route is always infeasible for the UAV. This paper proposed an efficient 3-D route planning algorithm for UAV based on Q-learning. The route constraints of UAV are efficiently used to guide the discretization of the planning space in the proposed algorithm, which not only reduces the scale of the resulted discrete planning problem, but also improves the feasibility of the resulted route for UAV. A Reward shaping mechanism, which is commonly used in reinforcement learning problem that can significantly improve the convergence property, is adopted to construct a more proper reward function. The simulation results of the typical 3-D route planning problem of UAV demonstrate that the proposed algorithm can efficiently address the 3-D route planning mission of UAV.
Key words：

Key words: unmanned aerial vehicle, three-dimensional route planning, heuristic information, route constraint, Q-learning

中图分类号: