VistaRAG: Toward Safe and Trustworthy Autonomous Driving Through Retrieval-Augmented Generation




Dai X, Guo C, Tang Y, Li H, Wang Y, Huang J, Tian Y, Xia X, Lv Y, Wang F

PublisherIEEE

2024

IEEE transactions on intelligent vehicles

IEEE Transactions on Intelligent Vehicles

9

4

4579

4582

2379-8858

DOIhttps://doi.org/10.1109/TIV.2024.3396450

https://ieeexplore.ieee.org/abstract/document/10518077



Autonomous driving based on foundation models has recently garnered widespread attention. However, the risk of hallucinations inherent in foundation models could compromise the safety and reliability of autonomous driving systems. This letter, as part of a series of reports from the Distributed/Decentralized Hybrid Workshop on Foundation/Infrastructure Intelligence (DHW-FII), aims to tackle these issues. We introduce VistaRAG, which integrates retrieval-augmented generation (RAG) technologies into autonomous driving systems based on foundation models, to address the inherent reliability challenges in decision-making. VistaRAG employs a dynamic retrieval mechanism to access highly relevant driving experience, real-time road network status, and other contextual information from external databases. This aids foundation models in informed reasoning and decision-making, thereby enhancing the safety and trustworthiness of foundation-model-based autonomous driving systems under complex traffic scenarios.



Last updated on 2025-27-01 at 19:28