About ADTRAN

ADTRAN is a global provider of analytics-enabled devices for delivering home and business communications and networking solutions. ADTRAN helps communications service providers around the world manage and scale services that connect people, places, and things to advance human progress. Whether rural or urban, domestic or international, telco or cable, enterprise or residential, ADTRAN solutions optimize existing technology infrastructures and create new, multi-gigabit platforms that leverage cloud economics, data analytics, machine learning and open ecosystems—the future of global networking.

The Challenge

ADTRAN was investigating ways to predict customer dissatisfaction using predictive Machine Learning algorithms based on historical data. They wanted to determine what patterns of device data indicate that a customer might become dissatisfied with their service. One of the challenges in doing so is that ADTRAN’s database structure was optimized to quickly retrieve up-to-the-minute device-level analytics data so that customer service representatives (CSRs) can effectively solve an individual customer’s problems. Given that per-second data volume into the database is very high, immediate redundant read access is critical. Such a structure is well-suited to their business model but presents challenges for putting together large-scale historical data for the machine learning model training.

Another barrier was in the data labeling as either “satisfied” or “unsatisfied.” ADTRAN does not have direct access to customer satisfaction data, as this data resides with their partner firms. In addition, customer churn data is sensitive and difficult to obtain, so ADTRAN needed an alternative solution. With all these challenges in mind, they began an ambitious research process and subsequent proof of concept to determine whether they could more accurately predict customer dissatisfaction.

Why Amazon Web Services

ADTRAN had already been using Amazon Web Services (AWS) for its Cassandra database cluster. Because of their positive experience and the importance of being able to interact smoothly with their existing infrastructure as they build further data analysis and usage technologies, they knew looking to AWS for solutions was a no-brainer.

Even though ADTRAN’s databases were already running on AWS, they wanted to leverage that data to improve the customer experience. While they had experience in architecting data for their core business, they needed help in architecting the AWS services that would be optimal to store historical data in a data lake for further processing. They also wanted help in assessing and leveraging the Amazon SageMaker for machine learning modeling and operation. Based on 1Strategy’s reputation for having AWS expertise and for delivering the highest quality, ADTRAN turned to 1Strategy for assistance with this proof of concept at the recommendation from AWS.

The Potential

1Strategy provided ADTRAN education about migrating to a data lake and provided support as ADTRAN performed their migration. In addition, 1Strategy demonstrated the utility that a data lake could provide them for future ML projects through Model Development data processing (diagram shown below). 1Strategy used AWS Glue to process sample device-level data sets from both gateways, such as home wi-fi routers, and gateway-connected devices, such as cell phones and computers. AWS Glue can process large datasets efficiently by applying data transformations. Because the volume of data was too large for rapid model development in SageMaker’s Jupyter notebook environment, the team used AWS Glue to perform schema regularization, feature engineering, time differentiation on cumulative quantities, and aggregation by time and customer.

In order to create a labeled dataset, the team chose to associate calls initiated to the call center as a marker for potential customer dissatisfaction. A second data path combined this data with the device-level data in a SageMaker notebook.

The team chose to model the time series data using a predictive maintenance model where the model uses data from a narrow historical time window to predict whether there will be a customer call in a forward-looking time window. Initially, the team modeled the data using a logistic regression with feature selection through Lasso regularization. The performance of this algorithm is shown in the confusion matrix below for a test sample of customers. The dark blue box in the upper left demonstrates that the algorithm correctly identifies satisfied callers who will not call a CSR. In contrast, the lower half of the matrix shows that the algorithm can correctly identify many of the customers who are about to initiate a call. However, the algorithm was not able to overcome the problem of class imbalance. Since satisfied customers greatly outnumber dissatisfied customers to start with, when the algorithm tags a customer as dissatisfied, it is still more likely that this customer is satisfied. Put another way, if you called all those customers to try and help, more often than not they would not need that help, so it is more effective to wait for their call.

Despite ultimately not being implemented, this project provided many valuable lessons along the way. Through this process, 1Strategy helped ADTRAN evaluate and plan for future potential machine learning projects. It demonstrated the need for differing data processing and storage architectures for the purpose of data analysis and model construction. The project also clarified some of the most difficult issues that machine learning faces in their environment, including class imbalance and data labeling.

About 1Strategy

1Strategy is an Amazon Web Services (AWS) Partner Network (APN) Premier Consulting Partner, focusing exclusively on AWS. 1Strategy helps businesses architect, migrate, and optimize their workloads on AWS, creating scalable, cost-effective, secure, and reliable solutions. 1Strategy also helps customers get real value from their data using comprehensive machine learning models and artificial intelligence. 1Strategy holds the AWS DevOps, Migration, Data & Analytics, and Machine Learning Competencies, and is a partner of the AWS Well Architected and Public Sector programs. 1Strategy was one of the initial ten AWS Partners globally who was qualified and authorized by AWS to conduct a Well-Architected Review and is among the top Well Architected partners in the AWS eco-system. With experts having deployed AWS solutions since 2007, 1Strategy is a leader in custom training—providing customers with the knowledge, tools, and best practices to manage those solutions over time. 1Strategy is a TEKsystems Global Services company with teams in Seattle and Salt Lake City, supporting customers throughout the US and across every vertical.