This thesis presents a toolkit of 17 user experience (UX) principles, which are categorized according to their relevance towards Explainable AI (XAI).
The goal of Explainable AI has been widely associated in literature with dimensions of comprehensibility, usefulness, trust, and acceptance. Moreover, authors in academia postulate that research should rather focus on the development of holistic explanation interfaces instead of single visual explanations. Consequently, the focus of XAI research should be more on potential users and their needs, rather than purely technical aspects of XAI methods. Considering these three impediments, the author of this thesis derives the assumption to bring valuable insights from the research area of User Interface (UI) and User Experience design into XAI research. Basically, UX is concerned with the design and evaluation of pragmatic and hedonic aspects of a user’s interaction with a system in some context.
These principles are taken into account in the subsequent prototyping of a custom XAI system called Brain Tumor Assistant (BTA). Here, a pre-trained EfficientNetB0 is used as a Convolutional Neural Network that can divide x-ray images of a human brain into four classes with an overall accuracy of 98%. To generate factual explanations, Local Interpretable Model-agnostic Explanations are subsequently applied as an XAI method. The following evaluation of the BTA is based on the so-called User Experience Questionnaire (UEQ) according to Laugwitz et al. (2008), whereby single items of the questionnaire are adapted to the specific context of XAI. Quantitative data from a study with 50 participants in each control and treatment group is used to present a standardized way of quantifying the dimensions of Usability and UX specifically for XAI systems. Furthermore, through an A/B test, evidence is presented that visual explanations have a significant (α=0.05) positive effect on the dimensions of attractiveness, usefulness, controllability, and trustworthiness. In summary, this thesis proves that explanations in computer vision not only have a significantly positive effect on trustworthiness, but also on other dimensions.
Contents
1 Introduction
2 Theoretical Foundation
2.1 Artificial Intelligence (AI)
2.2 Explainable Artificial Intelligence (XAI)
2.2.1 Explainability and Related Terms
2.2.2 Definition
2.2.3 Taxonomy of XAI Methods
2.3 User Experience (UX)
3 Methodology
3.1 Systematic Literature Research (SLR)
3.2 User-centered XAI Design
4 Prototyping and Evaluating an XAI Interface
4.1 Deriving Principles of User Experience
4.1.1 UX Principles in general
4.1.2 UX Principles according to XAI
4.2 Prototyping an XAI Interface for Computer Vision Tasks
4.2.1 Phase 1: Context
4.2.2 Phase 2: User
4.2.3 Phase 3: Solution
4.3 Evaluating an XAI Interface
4.3.1 Construction of an XAI-related Questionnaire
4.3.1.1 User Experience Questionnaire (UEQ)
4.3.1.2 Adapting the UEQ according to XAI
4.3.2 User Study
4.3.2.1 Design and Execution
4.3.2.2 General Results
4.3.2.3 Results from Quantitative Data Analysis
5 Conclusion
Research Objectives & Key Thematics
The primary objective of this thesis is to investigate how visual explanations within AI-based decision support systems (specifically for computer vision tasks like brain tumor detection) influence user experience dimensions, such as usability, trust, and acceptance. The thesis aims to establish a standardized method for quantifying these effects using a user-centered design approach and a validated questionnaire.
- Development of a toolkit comprising 17 UX principles categorized by their relevance to XAI systems.
- Prototyping of a custom XAI system (Brain Tumor Assistant - BTA) based on UCD phases.
- Application of Local Interpretable Model-agnostic Explanations (LIME) for generating visual explanations in medical image analysis.
- Quantitative evaluation of user experience through an adapted User Experience Questionnaire (UEQ) in a controlled A/B test study.
- Statistical assessment of the impact of visual explanations on user trust, attractiveness, controllability, and usefulness.
Excerpt from the Book
4.2.1 Phase 1: Context
As already described in chapter 3.2, the first step of the UCD process can be understood as the initial strategy of the tool [MZ21]. Thereby, the concrete context in which the user has to fulfill a certain task [PB09] as well as concrete objectives of the tool should be analyzed and defined at an early stage. This enables a structured development process regarding the feature requirements of the tool, which will be elaborated in the next phase [Som12].
The specific context, respectively use case, for which an XAI system is being developed is the detection of brain tumors in x-ray images of a human brain using a CNN. The Brain Tumor Assistant is a tool to support employees in their task of diagnosing brain tumors. The tool thus takes place in the professional environment of a user, as well as in high-stake decisions that have an impact on human life. Early detection and treatment of brain tumors is essential to prevent the spread of metastases and thus increase a patient’s chances of survival [SSA20]. It is estimated that between 70,000 and 400,000 U.S. citizens are diagnosed with brain metastases each year [LWA21]. Since radiologists are already outperformed by AI in the accuracy of their prognosis, the use of AI in x-ray diagnostics is of justifiably high relevance [Unk22]. Another reason for the author’s choice of context is that the subsequent chapter aims to quantify the dimensions of Usability and UX. As already described in chapter 2.3, for instance, Usability deals with the efficient, effective and satisfactory task performance of a user. Therefore, with regard to the further evaluation, it is important to choose a use case in which users are intrinsically motivated to use the BTA as an XAI system to solve their task.
Summary of Chapters
1 Introduction: Introduces the research motivation regarding XAI and its impact on end-user trust and usability in decision support systems.
2 Theoretical Foundation: Provides a comprehensive overview of AI, XAI, and UX definitions, including the design of explanation interfaces.
3 Methodology: Details the research methodology, specifically the Systematic Literature Research (SLR) and the user-centered design (UCD) approach.
4 Prototyping and Evaluating an XAI Interface: Describes the extraction of UX principles, the development of the BTA prototype, and the execution and analysis of the user study.
5 Conclusion: Summarizes the key research findings, discusses the impact of visual explanations on user experience, and highlights limitations for future research.
Keywords
Explainable Artificial Intelligence, XAI, User Experience, UX, Usability, Convolutional Neural Network, Brain Tumor Detection, Human-Computer Interaction, Interface Design, Quantitative User Study, User Experience Questionnaire, UEQ, LIME, Decision Support Systems, Visual Explanations.
Frequently Asked Questions
What is the core focus of this research?
The research focuses on closing the gap between purely technical XAI development and user-oriented design, specifically investigating how explanation interfaces affect the user experience of AI-based decision support systems in high-stakes fields like medicine.
Which domains are the central focus of the work?
The work focuses on the intersection of Explainable AI (XAI) and User Experience (UX) design, with a specific practical application in the field of radiology and brain tumor detection.
What is the primary objective of this thesis?
The primary goal is to determine if and how visual explanations in computer vision tasks positively influence specific user experience dimensions, such as trustworthiness, usefulness, controllability, and attractiveness.
Which scientific methodology is applied?
The thesis utilizes a Systematic Literature Research (SLR) to extract UX principles, followed by a user-centered design (UCD) approach to create a prototype, and finally, a quantitative A/B test study to empirically measure UX effects.
What is covered in the main body (Chapter 4)?
Chapter 4 details the extraction of 17 UX principles tailored for XAI, the prototyping of the "Brain Tumor Assistant" (BTA) using Figma, and the evaluation of user responses gathered through an adapted User Experience Questionnaire (UEQ).
Which keywords characterize this work?
The work is characterized by terms such as XAI, User Experience, Usability, Convolutional Neural Networks, Deep Learning, LIME, and Brain Tumor Detection.
How were visual explanations implemented in the prototype?
Visual explanations were implemented using LIME (Local Interpretable Model-agnostic Explanations), a perturbation-based method that segments X-ray images to identify and highlight relevant pixels that contributed most to a specific AI classification.
What was the key finding regarding visual explanations?
The study found that visual explanations have a highly significant positive effect on the dimension of trustworthiness, as well as significant positive impacts on usefulness, attractiveness, and controllability compared to an interface without visual explanations.
- Arbeit zitieren
- Georg Dedikov (Autor:in), 2023, Explainable AI and User Experience. Prototyping and Evaluating an UX-Optimized XAI Interface in Computer Vision, München, GRIN Verlag, https://www.grin.com/document/1356885
