Navigating Explainable AI in In Vitro Diagnostics: Compliance and Transparency Under European Regulations

The Role of Explainable AI in In Vitro Diagnostics Under European Regulations: AI is increasingly critical in healthcare, especially in vitro diagnostics (IVD). The European IVDR recognizes software, including AI and ML algorithms, as part of IVDs. This regulatory framework presents significant challenges for AI-based IVDs, particularly those that utilize DL techniques. These AI systems must perform accurately and provide explainable results to comply with regulatory requirements. Trustworthy AI is essential, as it must empower healthcare professionals to confidently use AI in decision-making, necessitating the development of explainable AI (xAI) methods. Tools like layer-wise relevance propagation can help visualize the elements of a neural network that contribute to specific outcomes, providing the necessary transparency.

The IVDR outlines rigorous criteria for developing and evaluating AI-based IVDs, including scientific validity, analytical performance, and clinical performance. As AI becomes more integrated into medical diagnostics, ensuring the transparency and traceability of these systems is crucial. Explainable AI addresses these needs by making the decision-making process of AI systems more understandable for medical professionals, which is critical in high-stakes environments like medical diagnostics. The focus will be on developing human-AI interfaces that blend AI’s computational power with human expertise, creating a synergy that enhances diagnostic accuracy and reliability.

Explainability and Scientific Validity in AI for In Vitro Diagnostics:

The IVDR describes scientific validity as the link between an analyte and a specific clinical condition or physiological state. When applying this to AI algorithms, the results must be explainable rather than simply produced by an opaque “black box” model. This distinction is important for validated diagnostic methods and AI algorithms supporting or replacing these methods. For example, an AI system designed to detect and quantify PD-L1 positive tumor cells must provide pathologists with a clear and understandable process. Similarly, in colorectal cancer survival prediction, AI-identified features must be explainable and supported by scientific evidence, requiring independent validation to ensure the results are trustworthy and accurate.

Explainability in Analytical Performance Evaluation for AI in IVDs:

In evaluating the analytical performance of AI in IVDs, it is crucial to ensure that AI algorithms accurately process input data across the full intended spectrum. This includes considering patient population, disease conditions, and scanning quality. Explainable AI (xAI) methods are key in defining valid input ranges and identifying when and why AI solutions may fail, particularly in data quality issues or artifacts. Proper data governance and a comprehensive understanding of training data are essential to avoid biases and ensure robust, reliable AI performance in real-world applications.

Explainability in Clinical Performance Evaluation for AI in IVDs:

Clinical performance evaluation of AI in IVDs assesses the AI’s ability to provide results relevant to specific clinical conditions. xAI methods are crucial in ensuring that AI supports decision-making effectively. These methods focus on making the AI’s decision process traceable, interpretable, and understandable for medical experts. The evaluation distinguishes between components that provide scientific validation and those that clarify medically relevant factors. Effective explainability requires static explanations and interactive, human-centered interfaces that align with experts’ needs, enabling deeper causal understanding and transparency in AI-assisted diagnoses.

Conclusion:

For AI solutions in IVDs to fulfill their intended purpose, they must demonstrate scientific validity, analytical performance, and, where relevant, clinical performance. Ensuring traceability and trustworthiness requires that explanations are reproducibly verifiable by different experts and are technically interoperable and understandable. xAI methods address critical questions: why the AI solution works when it can be applied and why it produces specific results. In the biomedical field, where AI has vast potential, xAI is crucial for regulatory compliance and empowering healthcare professionals to make informed decisions. The paper highlights the importance of explainability and usability in ensuring the validity and performance of AI-based IVDs.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here

Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.