Problem Statement:
A major E-commerce company in the USA wanted to develop an anomaly detection algorithm for their web-infrastructure system to identify and isolate reasons for detecting violations in response times encountered by customers and predict them 5 minutes ahead of time.
Solution:
To solve the problem, the team applied multiple data science techniques including random forest, deep learning, and clustering. After investigation, random forest produced the best possible result in being able to predict response time violations 5 minutes ahead of time with 93% accuracy.
The challenges of the anomaly detection case study included dealing with a large amount of unclean and unstructured data, difficulty in identifying observable patterns, and inconsistent response time violations. The lack of set definitions for anomalies, varying server instances, and the need to design the response time of the server added to the complexity of the task.
Benefits:
The implementation of the anomaly detection algorithm led to better customer experiences, increased support system efficiencies, and reduced customer churn rate.
