Um processo para avaliação e gerenciamento de mudanças de modelos de aprendizado de máquina aplicado a área da saúde

Abstract

Fostered by hardware and software advances, Machine Learning (ML) started to ramp up exponentially in the last few decades, and has become instrumental for advancing the work in the most varied areas of knowledge. Though generally restricted to controlledspace experiments, over previously obtained and curated data samples, results have been outstanding, which gave rise to such levels of popularity for ML applications that it’s hard to find an area of human knowledge left untouched by Machine Learning. In such context, establishing minimum performance guarantees over unknown, real-world data, becomes paramount, especially in Healthcare applications, where errors can lead to life-threatening situations. There’s an ML discipline, called Machine Learning Operations (or MLOps, for short), which concerns itself with ML Models’ lifecycle management, from conception to deployment in production (real-world) environments, including monitoring its real-world behavior. Once deployed, models are subject to performance decay issues, such as drift, which has motivated recent studies on continual learning and Continuous Monitoring of ML models. The present work proposes a process for ML model evaluation designed for Healthcare applications running on real-world data. To that end, a conducted Systematic Literature Review (SLR) aimed at determining the state-of-the-art techniques and methods for ML evaluation and a case study applied the proposed process to ML models in an oncologic ICU. The Case Study produced positive outcomes in establishing a feedback loop for models in use against real-world data.