Human-in-the-Loop and Generative AI Dilemma: A Hybrid Strategy for Effective Customer Service in Enterprise CRM
List of Authors
Naga Sai Mrunal Vuppala, Sri Hari Deep Kolagani
Keyword
Large Language Models; Customer Relationship Management; Fine-tuning; Enterprise AI; Generative AI (GenAI).
Abstract
Customer relationship management (CRM) systems are changing more than ever before, thanks to rapid advances in artificial intelligence space. Generative AI (GenAI) now is capable to answer most routine customer questions automatically. Human agents step in only for complex cases that need expert judgment, empathy, or careful handling due to rules and regulations. However, companies still find it difficult to make AI and people work in harmony. This research introduces and tests a hybrid CRM approach that combines GenAI with a clear human-in-the-loop (HITL) process. We use the NATCS dataset, which contains real customer service conversations from 2023 in fields like banking, finance, health, travel, and insurance. We fine-tune an AI assistant, set up rules for when humans should take over based on uncertainty in the conversation, and measure results using both data and human feedback. Our hybrid system makes customer service faster, empathetic and simpler. It also provides streamlined guidelines for managing risks, security, and data, meeting high academic standards. The results show that this hybrid approach increases first-contact resolution by 27%, cuts average handling time by 34%, and boosts customer satisfaction by 22% compared to using AI alone. It also reduces the number of times human agents need to speak by 41% compared to traditional methods.