Evaluating the ROI of Sentiment-Aware Chatbot Routing: An Empirical Study of Social Media Support Automation Using Machine Learning

• Abiodun Okunola

  Ladoke Akintola University Technology

  Author

Social media platforms have become primary channels for customer support, yet traditional routing systems treat all inquiries uniformly, failing to account for the emotional state and urgency conveyed in customer messages. This oversight leads to escalated complaints, customer churn, and operational inefficiencies. While prior research has demonstrated the technical feasibility of sentiment analysis in social media contexts, no validated framework exists that quantifies the financial return on investment (ROI) of sentiment-aware routing systems. This study addresses this gap by developing and empirically evaluating a sentiment-aware chatbot routing framework for social media support automation. Using a dataset of over 6,500 Twitter conversations with Amazon's @AmazonHelp support account, we implement and compare multiple machine learning classifiers for sentiment detection, achieving an optimal F1-score of 89.4% using a hybrid K-Nearest Neighbor and Support Vector Machine approach. We then simulate routing performance against baseline keyword-based and random routing methods across 100,000 synthetic support interactions. Results demonstrate that sentiment-aware routing reduces call escalations by 17%, improves customer satisfaction from 3.4 to 4.1 on a 5-point scale, and delivers an estimated ROI of 340% over 12 months through reduced agent handling time and improved retention. The framework provides practitioners with a replicable methodology for quantifying automation investments and contributes to the emerging literature on emotion-aware operations research.

• pdf

Copyright (c) 2026 Abiodun Okunola (Author)

This work is licensed under a Creative Commons Attribution 4.0 International License.