EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI) FOR IMPROVING TRUST AND TRANSPARENCY IN HEALTHCARE DECISION SUPPORT SYSTEMS

Abstract

The use of AI in healthcare decision support systems has improved accuracy in diagnostics and greater efficiency in clinical practice. The adoption of these systems is hindered by the “black-box” problem. For systems that work in domains that make decisions with life and death impact, the inability to explain the reasoning of the decision makes these systems hard to trust. There is also the problem of accountability and the ethical burden. Explainable AI (XAI) provides systems with the ability to explain their reasoning in a comprehensible format. The use of XAI in HDSS provides trust to users of the system, because of the transparency and the consideration of the methodologies of XAI like model agnostic systems, and other methodologies, in the design of the systems for the clinical users. The use of XAI promotes and enables regulatory approval for the system by providing the ability to explain the decision in a comprehensible and reasoned way. The paper looks at the challenges of XAI like the accuracy vs explainability of the system, and the other challenges of the regulatory approval of the system for its use in clinical settings.