Desenvolvimento de um sistema multi-agente baseado em inteligências artificiais generativas para avaliação de qualidade em atendimento ao cliente

Abstract

This work presents the development and validation of a multi-agent system based on Large Language Models (LLMs) for quality assessment in customer service, aiming to overcome the limitations of traditional monitoring methods. The system implements an innovative architecture composed of five specialized agents: Supervisor, Quality Monitor, Operator, Judge, and Reporting Analyst, each responsible for specific aspects of the evaluation. The methodology employed the CrewAI framework for multi-agent system development, using the GPT-4o-mini model as a base, and implemented a web interface in Streamlit for user interaction. The system validation was performed through tests with a real dataset from an energy sector company, comparing its performance with specialized human evaluations and other language models. The results showed 90% to 100% agreement with human evaluations, significantly higher than the 50% achieved by approaches based on isolated models. The system maintained a consistent average time of 6 minutes per analysis, representing a reduction of up to 52% in the time required compared to traditional monitoring. The solution also demonstrated exceptional economic viability, with a 95.4% reduction in cost per analysis while maintaining high quality in evaluations. The quality of generated reports was validated by independent evaluators, obtaining an average score of 32.5/35. The developed system establishes a new paradigm for quality assessment in customer service, combining operational efficiency, economic viability, and consistency in analyses, allowing quality monitors to focus on higher value-added strategic activities.