Certification and Trustworthy AI

Stochastic Application Domain Definition for Functional Trustworthiness Certification of AI Systems

---

Abstract

< >As Artificial Intelligence (AI) systems are increasingly deployed in safety-critical and societally consequential contexts, the question of how to evaluate their performance in a trustworthy and interpretable manner becomes increasingly important. Within the European Union, this issue is reflected in the AI Act, which requires training, validation, and testing datasets to be relevant and sufficiently representative with respect to the system's intended purpose. This raises a fundamental technical question: representative of what population of situations?< >From a statistical perspective, performance metrics such as error rates or expected losses are always defined with respect to a probability distribution. We refer to this distribution as the Application Domain (AD). In practice, however, the AD of real-world AI systems is rarely known in explicit mathematical form and must instead be characterized operationally through the procedures by which valid samples are generated or selected.< >To address this problem, we introduce the Stochastic Application Domain Definition (SADD), a textual specification of the sampling protocol that induces the distribution under which performance claims are intended to hold. The SADD links intended use, dataset construction, and statistical testing by making the underlying data-generation assumptions explicit. We formalize the notion of protocol-induced distributions, discuss how SADDs guide feasible sampling procedures, contrast the approach with qualitative domain descriptions such as Operational Design Domains, and examine implications for the certification of AI systems.

How to Cite:

Schmid, S., Brune, B., Aufreiter, A., Gruber, L., Schweighofer, K., Stadlbauer, X., Doms, T. & Nessler, B., (2026) “Stochastic Application Domain Definition for Functional Trustworthiness Certification of AI Systems”, Proceedings of the Austrian Symposium on AI, Robotics, and Vision 3(1), 265-281.