Reasoning About Emergent Behaviors at Runtime

Abstract

As Autonomous Vehicles (AVs) become more common in the market, the development of new technologies and solutions for them advances quickly. However, there still are safety concerns regarding the increasing presence of AVs on public roads.For example, the California Department of Motor Vehicles reported an approximate average of 3 disengagement events for each AV under test in 2021 and 1 event for every 1,500 driven miles. Considering that a vehicle drives, on average, 15,000 miles per year, an AV would have ten such incidents annually. With this in mind, work needs to be done to enable AVs to handle unexpected situations (i.e., emergent behaviors) that can arise while driving, increasing the safety for both humans and AVs on the road. This dissertation discusses possible approaches to improve the detection and reasoning of emergent behaviors at runtime.

First, it describes SAFER, a framework to monitor individual systems using runtime verification alongside machine learning anomaly detection to detect emergent behaviors at runtime. SAFER achieves an average F1-score of 83% over four different example systems, showcasing its efficacy to be on par with related work while being applicable to different classes of applications. Next, it discusses DRIVE, a framework for monitoring collective systems that can work together to find global anomalies that affect collective safety by identifying violations in local properties. Throughout a state-of-the-art inspired truck platooning case study, DRIVE can detect all safety property violations, with most queries resolved under 1 ms and a maximum latency of 11 ms per query, demonstrating its promise. Lastly, it presents LOCoCAT, a low-overhead framework for emergent behavior reasoning to classify anomaly types based on vehicular data. LOCoCAT achieves an F1-score of up to 99.16% within the first 50 ms of the anomaly, allowing the system to react quickly. These three techniques can be combined to allow future AVs to operate more safely on the road.