Journal of Shanghai Jiaotong University >
Decision-Making Method of Intelligent Vehicles: A Survey
Received date: 2020-11-23
Online published: 2021-06-08
Combined with the current research status of the intelligent vehicle decision-making methods at home and abroad, this paper classifies and summarizes decision-making methods from four aspects: decision input and output, environment interaction, and algorithm types. Besides, it analyzes their advantages and disadvantages, and evaluates applicable scenarios. Moreover, it surveyes the common data sets and current evaluation standards which are used for decision-making researches. Furthermore it discusses the technical difficulties faced by current decision-making methods and future development trends.
Key words： intelligent vehicle; decision-making method; evaluation standard; dataset
HU Yikai, WANG Chunxiang, YANG Ming . Decision-Making Method of Intelligent Vehicles: A Survey[J]. Journal of Shanghai Jiaotong University, 2021 , 55(8) : 1035 -1048 . DOI: 10.16183/j.cnki.jsjtu.2020.387
| [1] | 胡兵, 杨明, 郭林栋, 等. 基于地面快速鲁棒特征的智能车全局定位方法[J]. 上海交通大学学报, 2019, 53(2):203-208. |
| [1] | HU Bing, YANG Ming, GUO Lindong, et al. Global localization for intelligent vehicles using ground SURF[J]. Journal of Shanghai Jiao Tong University, 2019, 53(2):203-208. |
| [2] | 杜明博, 梅涛, 陈佳佳, 等. 复杂环境下基于RRT的智能车辆运动规划算法[J]. 机器人, 2015, 37(4):443-450. |
| [2] | DU Mingbo, MEI Tao, CHEN Jiajia, et al. RRT-based motion planning algorithm for intelligent vehicle in complex environments[J]. Robot, 2015, 37(4):443-450. |
| [3] | 陈佳佳. 城市环境下无人驾驶车辆决策系统研究[D]. 合肥: 中国科学技术大学, 2014. |
| [3] | CHEN Jiajia. Research on decision making system of autonomous vehicle in urban environments[D]. Hefei: University of Science and Technology of China, 2014. |
| [4] | SCHWARTING W, ALONSO-MORA J, RUS D. Planning and decision-making for autonomous vehicles[J]. Annual Review of Control, Robotics, and Autonomous Systems, 2018, 1(1):187-210. |
| [5] | 熊璐, 康宇宸, 张培志, 等. 无人驾驶车辆行为决策系统研究[J]. 汽车技术, 2018(8):1-9. |
| [5] | XIONG Lu, KANG Yuchen, ZHANG Peizhi, et al. Research on behavior decision-making system for unmanned vehicle[J]. Automobile Technology, 2018(8):1-9. |
| [6] | GONZÁLEZ D S, GARZÓN M, DIBANGOYE J S, et al. Human-like decision-making for automated driving in highways[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 2087-2094. |
| [7] | NOH S, AN K. Decision-making framework for automated driving in highway environments[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(1):58-71. |
| [8] | XIONG G M, KANG Z Y, LI H, et al. Decision-making of lane change behavior based on RCS for automated vehicles in the real environment[C]//2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 1400-1405. |
| [9] | NISHI T, DOSHI P, PROKHOROV D. Merging in congested freeway traffic using multipolicy decision making and passive actor-critic learning[J]. IEEE Transactions on Intelligent Vehicles, 2019, 4(2):287-297. |
| [10] | KANG K, RAKHA H A. A repeated game freeway lane changing model[J]. Sensors, 2020, 20(6):1554. |
| [11] | KANG K, RAKHA H A. Game theoretical approach to model decision making for merging maneuvers at freeway on-ramps[J]. Transportation Research Record Journal of the Transportation Research Board, 2017, 2623(1):19-28. |
| [12] | GUO J Q, CHENG S L, LIU Y. Merging and diverging impact on mixed traffic of regular and autonomous vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(3):1639-1649. |
| [13] | HOEL C J, WOLFF K, LAINE L. Automated speed and lane change decision making using deep reinforcement learning[C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui, HI, USA: IEEE, 2018: 2148-2155. |
| [14] | ALIZADEH A, MOGHADAM M, BICER Y, et al. Automated lane change decision making using deep reinforcement learning in dynamic and uncertain highway environment[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 1399-1404. |
| [15] | LI T T, WU J P, CHAN C Y. Evolutionary learning in decision making for tactical lane changing[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 19211185. |
| [16] | CODEVILLA F, MÜLLER M, LÓPEZ A, et al. End-to-end driving via conditional imitation learning[C]//2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane, QLD, Australia: IEEE, 2018: 4693-4700. |
| [17] | SANTANA E, HOTZ G. Learning a driving simulator[EB/OL]. (2016-08-01) [2020-06-14]. https://www.researchgate.net/publication/305809416_Learning_a_Driving_Simulator . |
| [18] | YUAN W, YANG M, WANG C X, et al. Longitudinal and lateral coupling model based end-to-end learning for lane keeping of self-driving cars[C]//International Conference on Cognitive Systems and Signal Processing. Beijing: ICCSIP, 2019: 425-436. |
| [19] | YUAN W, YANG M, LI H, et al. End-to-end learning for high-precision lane keeping via multi-state model[J]. CAAI Transactions on Intelligence Technology, 2018, 3(4):185-190. |
| [20] | YU H, YANG S, GU W H, et al. Baidu driving dataset and end-to-end reactive control model[C]//2017 IEEE Intelligent Vehicles Symposium (IV). Los Angeles, CA, USA: IEEE, 2017: 341-346. |
| [21] | BOJARSKI M, YERES P, CHOROMANSKA A, et al. Explaining how a deep neural network trained with end-to-end learning steers a car[EB/OL]. (2017-04-14) [2020-06-14]. https://www.researchgate.net/publication/316505624_Explaining_How_a_Deep_Neural_Network_Trained_with_End-to-End_Learning_Steers_a_Car . |
| [22] | IBERRAKEN D, ADOUANE L, DENIS D. Safe autonomous overtaking maneuver based on inter-vehicular distance prediction and multi-level Bayesian decision-making[C]//IEEE Conference on Intelligent Transportation Systems. Maui, HI, USA: IEEE, 2018: 18308831. |
| [23] | TRAM T, BATKOVIC I, ALI M, et al. Learning when to drive in intersections by combining reinforcement learning and model predictive control[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 19194092. |
| [24] | ORZECHOWSKI P F, BURGER C, LAUER M. Decision-making for automated vehicles using a hierarchical behavior-based arbitration scheme[C]//2020 IEEE Intelligent Vehicles Symposium (IV). Las Vegas, NV, USA: IEEE, 2020: 20286743. |
| [25] | WANG X, WU J Q, GU Y L, et al. Human-like maneuver decision using LSTM-CRF model for on-road self-driving[C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui, HI, USA: IEEE, 2018: 210-216. |
| [26] | 成英, 高利, 陈雪梅, 等. 有人与无人驾驶车辆交叉口驾驶博弈模型[J]. 北京理工大学学报, 2019, 39(9):938-943. |
| [26] | CHENG Ying, GAO Li, CHEN Xuemei, et al. A driving game model for manned and unmanned vehicles at intersection[J]. Transactions of Beijing Institute of Technology, 2019, 39(9):938-943. |
| [27] | CHEN Z L, HUANG X M. End-to-end learning for lane keeping of self-driving cars[C]//2017 IEEE Intelligent Vehicles Symposium (IV). Los Angeles, CA, USA: IEEE, 2017: 1856-1860. |
| [28] | LI N, OYLER D W, ZHANG M X, et al. Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems[J]. IEEE Transactions on Control Systems Technology, 2018, 26(5):1782-1797. |
| [29] | CUNNINGHAM A G, GALCERAN E, EUSTICE R M, et al. MPDM: Multipolicy decision-making in dynamic, uncertain environments for autonomous driving[C]//2015 IEEE International Conference on Robotics and Automation (ICRA). Seattle, WA, USA: IEEE, 2015: 1670-1677. |
| [30] | SCHWARTING W, PASCHEKA P. Recursive conflict resolution for cooperative motion planning in dynamic highway traffic[C]//17th International IEEE Conference on Intelligent Transportation Systems (ITSC). Qingdao: IEEE, 2014: 1039-1044. |
| [31] | LENZ D, KESSLER T, KNOLL A. Tactical cooperative planning for autonomous highway driving using Monte-Carlo Tree Search[C]//2016 IEEE Intelligent Vehicles Symposium (IV). Gothenburg, Sweden: IEEE, 2016: 447-453. |
| [32] | CUNNINGHAM A G, GALCERAN E, MEHTA D, et al. MPDM: Multi-policy decision-making from autonomous driving to social robot navigation[M]//WASCHL H, KOLMANOVSKY I, WILLEMS F. Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions. Berlin, Germany: Springer, 2019: 201-223. |
| [33] | CHEN J Y, TANG C, XIN L, et al. Continuous decision making for on-road autonomous driving under uncertain and interactive environments[C]//2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 1651-1658. |
| [34] | WEI J Q, DOLAN J M, LITKOUHI B. Autonomous vehicle social behavior for highway entrance ramp management[C]//2013 IEEE Intelligent Vehicles Symposium (IV). Gold Coast, QLD, Australia: IEEE, 2013: 201-207. |
| [35] | HOERMANN S, STUMPER D, DIETMAYER K. Probabilistic long-term prediction for autonomous vehicles[C]//2017 IEEE Intelligent Vehicles Symposium (IV). Piscataway, NJ, USA: IEEE, 2017: 237-243. |
| [36] | DING W C, ZHANG L, CHEN J, et al. Safe trajectory generation for complex urban environments using spatio-temporal semantic corridor[J]. IEEE Robotics and Automation Letters, 2019, 4(3):2997-3004. |
| [37] | 武历颖. 无人驾驶汽车环境信息提取及运动决策方法研究[D]. 西安: 长安大学, 2016. |
| [37] | WU Liying. Research on environmental information extraction and movement decision-making method of unmanned vehicle[D]. Xi’an: Changan University, 2016. |
| [38] | LEURENT E, MERCAT J. Social attention for autonomous decision-making in dense traffic[EB/OL]. (2019-11-27) [2020-06-14]. https://arxiv.org/abs/1911.12250 . |
| [39] | MEHTA A, SUBRAMANIAN A. Learning end-to-end autonomous driving using guided auxiliary supervision[EB/OL]. (2018-08-30) [2020-06-14]. https://www.researchgate.net/publication/327335968_Learning_End-to-end_Autonomous_Driving_using_Guided_Auxiliary_Supervision . |
| [40] | XU J X, LUO Q, XU K C, et al. An automated learning-based procedure for large-scale vehicle dynamics modeling on Baidu Apollo platform[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ, USA: IEEE, 2019: 5049-5056. |
| [41] | SUTTON R S, BARTO A G. Reinforcement learning: An introduction[J]. IEEE Transactions on Neural Networks, 1998, 9(5):1054. |
| [42] | PIETER A, ANDREW Y. Apprenticeship learning via inverse reinforcement learning[EB/OL]. (2004-09-01) [2020-06-14]. https://www.researchgate.net/publication/2947330_Apprenticeship_Learning_via_Inverse_Reinforcement_Learning . |
| [43] | BOJARSKI M, DEL TESTA D, DWORAKOWSKI D, et al. End to end learning for self-driving cars[EB/OL]. (2016-04-01)[2020-06-15]. https://www.researchgate.net/publication/301648615_End_to_End_Learning_for_Self-Driving_Cars . |
| [44] | XU H Z, GAO Y, YU F, et al. End-to-end learning of driving models from large-scale video datasets[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 3530-3538. |
| [45] | HECKER S, DAI D X, GOOL L. End-to-end learning of driving models with surround-view cameras and route planners[C]//European Conference on Computer Vision. Glasgow, UK: Springer, 2018: 449-468. |
| [46] | GAO Z H, SUN T J, XIAO H W. Decision-making method for vehicle longitudinal automatic driving based on reinforcement Q-learning[J]. International Journal of Advanced Robotic Systems, 2019, 16(3):172988141985318. |
| [47] | YE Y J, ZHANG X H, SUN J. Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment[J]. Transportation Research Part C: Emerging Technologies, 2019, 107:155-170. |
| [48] | 郑睿. 基于增强学习的无人车辆智能决策方法研究[D]. 长沙: 国防科学技术大学, 2013. |
| [48] | ZHENG Rui. Reinforcement learning-based intelligent decision-making methods for unmanned vehicles[D]. Changsha: National University of Defense Technology, 2013. |
| [49] | YUAN W, YANG M, HE Y S, et al. Multi-reward architecture based reinforcement learning for highway driving policies[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 19194107. |
| [50] | QIAO Z Q, MUELLING K, DOLAN J M, et al. Automatically generated curriculum based reinforcement learning for autonomous vehicles in urban environment[C]//2018 IEEE Intelligent Vehicles Symposium (IV). Piscataway, NJ, USA: IEEE, 2018: 1233-1238. |
| [51] | BOUTON M, NAKHAEI A, FUJIMURA K, et al. Cooperation-aware reinforcement learning for merging in dense traffic[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Changshu: IEEE, 2019: 3441-3447. |
| [52] | KAUSHIK M, PRASAD V, KRISHNA K M, et al. Overtaking maneuvers in simulated highway driving using deep reinforcement learning[C]//2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 1885-1890. |
| [53] | FERDOWSI A, CHALLITA U, SAAD W, et al. Robust deep reinforcement learning for security and safety in autonomous vehicle systems[C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Piscataway, NJ, USA: IEEE, 2018: 307-312. |
| [54] | MA X B, CAMPBELL D K, KOCHENDERFER M J. Improved robustness and safety for autonomous vehicle control with adversarial reinforcement learning[C]//2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 18168097. |
| [55] | SHARIFZADEH S, CHIOTELLIS I, TRIEBEL R, et al. Learning to drive using inverse reinforcement learning and deep Q-networks[EB/OL]. (2016-12-12) [2020-06-14]. https://arxiv.org/abs/1612.03653 . |
| [56] | CHEN X L, MONFORT M, ZIEVART B D, et al. Adversarial inverse optimal control for general imitation learning losses and embodiment transfer[EB/OL]. (2016-06-01)[2020-06-14]. https://scholar.google.com.hk/scholar?hl=zh-CN&as_sdt=0%2C5&q=Adversarial+Inverse+Optimal+Control+for+General+Imitation+Learning+Losses+and+Embodiment+Transfer&btnG= . |
| [57] | MAJUMDAR A, SINGH S, MANDLEKAR A, et al. Risk-sensitive inverse reinforcement learning via coherent risk models[EB/OL]. [2020-06-14]. http://www.roboticsproceedings.org/rss13/p69.pdf . |
| [58] | WULFMEIER M, ONDRUSKA P, POSNER I. Maximum entropy deep inverse reinforcement learning[EB/OL]. (2015-07-17) [2020-06-14]. https://arxiv.org/abs/1507.04888 . |
| [59] | AHN H, BERNTORP K, DI CAIRANO S. Reachability-based decision making for city driving[C]//2018 Annual American Control Conference (ACC). Milwaukee, WI, USA: IEEE, 2018: 3203-3208. |
| [60] | BOUTON M, NAKHAEI A, FUJIMURA K, et al. Safe reinforcement learning with scene decomposition for navigating complex urban environments[C]//2019 IEEE Intelligent Vehicles Symposium (IV). Paris, France: IEEE, 2019: 1469-1476. |
| [61] | 陈雪梅, 田赓, 苗一松, 等. 城市环境下无人驾驶车辆驾驶规则获取及决策算法[J]. 北京理工大学学报, 2017, 37(5):491-496. |
| [61] | CHEN Xuemei, TIAN Geng, MIAO Yisong, et al. Driving rule acquisition and decision algorithm to unmanned vehicle in urban traffic[J]. Transactions of Beijing Institute of Technology, 2017, 37(5):491-496. |
| [62] | NISHI T, DOSHI P, PROKHOROV D. Merging in congested freeway traffic using multipolicy decision making and passive actor-critic learning[J]. IEEE Transactions on Intelligent Vehicles, 2019, 4(2):287-297. |
| [63] | ALEXIADIS V, COLYAR J, HALKIAS J, et al. The next generation simulation program[J]. Institute of Transportation Engineers. ITE Journal, 2004, 74(8):22. |
| [64] | LI G, FANG S, MA J X, et al. Modeling merging acceleration and deceleration behavior based on gradient-boosting decision tree[J]. Journal of Transportation Engineering, Part A: Systems, 2020, 146(7):1-9. |
| [65] | LI G, PAN Y Y, YANG Z, et al. Modeling vehicle merging position selection behaviors based on a finite mixture of linear regression models[J]. IEEE Access, 2019, 7:158445-158458. |
| [66] | BIN AL ISLAM S M A, ABDUL AZIZ H M, HAJBABAIE A. Stochastic gradient-based optimal signal control with energy consumption bounds[EB/OL]. (2020-03-18) [2020-06-14]. https://scholar.google.com.hk/scholar?hl=zh-CN&as_sdt=0%2C5&q=Stochastic+Gradient-Based+Optimal+Signal+Control+With+Energy+Consumption+Bounds&btnG= . |
| [67] | LI G, CHENG J. Exploring the effects of traffic density on merging behavior[J]. IEEE Access, 2019, 7:51608-51619. |
| [68] | KRAJEWSKI R, BOCK J, KLOEKER L, et al. The highD dataset: A drone dataset of naturalistic vehicle trajectories on German highways for validation of highly automated driving systems[C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui, HI, USA: IEEE, 2018: 18309040. |
| [69] | ZHAN W, SUN L, WANG D, et al. Interaction dataset: An international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps[EB/OL]. (2019-09-30) [2020-06-14]. https://arxiv.org/abs/1910.03088 . |
| [70] | HOUSTON J, ZUIDHOF G, BERGAMINI L, et al. One thousand and one hours: Self-driving motion Prediction dataset[EB/OL]. (2020-06-25) [2020-08-14]. https://arxiv.org/abs/2006.14480 . |
| [71] | KIM H, KIM D, KIM G, et al. Multi-head attention based probabilistic vehicle trajectory prediction[C]//2020 IEEE Intelligent Vehicles Symposium (IV). Las Vegas, NV, USA: IEEE, 2020: 1720-1725. |
| [72] | RUDENKO A, PALMIERI L, HERMAN M, et al. Human motion trajectory prediction: A survey[J]. The International Journal of Robotics Research, 2020, 39(8):895-935. |
| [73] | ETESAMI J, GEIGER P. Causal transfer for imitation learning and decision making under sensor-shift[C]// Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(6):10118-10125. |
| [74] | MESSAOUD K, YAHIAOUI I, VERROUST-BLONDET A, et al. Relational recurrent neural networks for vehicle trajectory prediction[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 19193897. |
| [75] | WANG X, NAIR S, ALTHOFF M. Falsification-based robust adversarial reinforcement learning[EB/OL]. (2020-07-01) [2020-06-14]. https://arxiv.org/abs/2007.00691 . |
| [76] | BHATTACHARYYA R P, SENANAYAKE R, BROWN K, et al. Online parameter estimation for human driver behavior prediction[C]//2020 American Control Conference (ACC). Denver, CO, USA: IEEE, 2020: 19830660. |
| [77] | QIAO Z Q, ZHAO J, ZHU J, et al. Human driver behavior prediction based on UrbanFlow[C]//2020 IEEE International Conference on Robotics and Automation (ICRA). Paris, France: IEEE, 2020: 10570-10576. |
| [78] | MESSAOUD K, YAHIAOUI I, VERROUST-BLONDET A, et al. Attention based vehicle trajectory prediction[J]. IEEE Transactions on Intelligent Vehicles, 2021, 6(1):175-185. |
| [79] | SUN L T, ZHAN W, HU Y P, et al. Interpretable modelling of driving behaviors in interactive driving scenarios based on cumulative prospect theory[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 4329-4335. |
| [80] | HU Y P, ZHAN W, TOMIZUKA M. Scenario-transferable semantic graph reasoning for interaction-aware probabilistic prediction[EB/OL]. (2020-04-01)[2020-06-14]. https://www.researchgate.net/publication/340500370_Scenario-Transferable_Semantic_Graph_Reasoning_for_Interaction-Aware_Probabilistic_Prediction . |
| [81] | SHU K Q, YU H L, CHEN X X, et al. Autonomous driving at intersections: A critical-turning-point approach for left turns[C]//2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). Rhodes, Greece: IEEE, 2020: 1-6. |
| [82] | SHU K, YU H L, CHEN X X, et al. Autonomous driving at intersections: A critical-turning-point approach for left turns[C]//2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). Rhodes, Greece: IEEE, 2020: 20303092. |
| [83] | FENG D, HAASE-SCHÜTZ C, ROSENBAUM L, et al. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(3):1341-1360. |
| [84] | CHANDRA R, GUAN T, PANUGANTI S, et al. Forecasting trajectory and behavior of road-agents using spectral clustering in graph-LSTMs[J]. IEEE Robotics and Automation Letters, 2020, 5(3):19741183. |
| [85] | SALZMANN T, IVANOVIC B, CHAKRAVARTY P, et al. Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data[C]//European Conference on Computer Vision-ECCV 2020. Glasgow, UK: Springer, 2020: 683-700. |
| [86] | GADD M, DE MARTINI D, MARCHEGIANI L, et al. Sense-assess-eXplain (SAX): Building trust in autonomous vehicles in challenging real-world driving scenarios[C]//2020 IEEE Intelligent Vehicles Symposium (IV). Las Vegas, NV, USA: IEEE, 2020: 150-155. |
/
| 〈 |
|
〉 |
