![[2511.14853] Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2511.14853] Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems
Summary
This paper presents a probabilistic method to measure the representativeness of scenario suites for autonomous systems, focusing on ensuring safety and trustworthiness in AI applications.
Why It Matters
As autonomous systems like self-driving cars become more prevalent, ensuring their safety through robust training datasets is critical. This research addresses the challenge of quantifying how well these datasets reflect real-world conditions, which is essential for developing reliable AI systems.
Key Takeaways
- Introduces a probabilistic approach to measure dataset representativeness.
- Focuses on the importance of operational design domains for AI safety.
- Utilizes imprecise Bayesian methods to handle uncertainty in data.
- Compares scenario suites against inferred target operational domains.
- Provides interval-valued estimates of representativeness rather than single values.
Computer Science > Artificial Intelligence arXiv:2511.14853 (cs) [Submitted on 18 Nov 2025 (v1), last revised 16 Feb 2026 (this version, v2)] Title:Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems Authors:Robab Aghazadeh Chakherlou, Siddartha Khastgir, Xingyu Zhao, Jerein Jeyachandran, Shufeng Chen View a PDF of the paper titled Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems, by Robab Aghazadeh Chakherlou and 4 other authors View PDF HTML (experimental) Abstract:Assuring the trustworthiness and safety of AI systems, e.g., autonomous vehicles (AV), depends critically on the data-related safety properties, e.g., representativeness, completeness, etc., of the datasets used for their training and testing. Among these properties, this paper focuses on representativeness-the extent to which the scenario-based data used for training and testing, reflect the operational conditions that the system is designed to operate safely in, i.e., Operational Design Domain (ODD) or expected to encounter, i.e., Target Operational Domain (TOD). We propose a probabilistic method that quantifies representativeness by comparing the statistical distribution of features encoded by the scenario suites with the corresponding distribution of features representing the TOD, acknowledging that the true TOD distribution is unknown, as it can only be inferred from limited data. We apply an imprecise Bayesian method to handle...
