Case Studies

Mobile Network Operator (MNO) Credit Scoring and Automated Underwriting

March 28, 2017

Results

  • MNO emerging markets project using GSM, Airtime Credit and transaction data
  • High predictive results for mobile lending functionality (AUC 0.78-0.91)
  • Project duration, 8 weeks from receipt of data

Is it possible to use MNO data on its own to predict customer behavior, credit risk, probability of default, fraud and other attributes? Is MNO data sufficient to do this reliably and accurately?

Yes.

The following case study shows that MNO data is both predictive and can be used in isolation in markets where other data sources, such traditional credit bureau and related banking data, are limited or absent. The results show a level of accuracy and stability, sufficient to support profitable, mobile financial services. The project also shows the resource and time required using Zoral Platform components.

Project summary

Project goals

The main goal was to create and benchmark a pilot, automated, consumer lending platform to integrate with the client’s Mobile Financial Services (MFS) architecture, including,

Methodology

The diagram below shows a general approach to tuning the ZML models selected:

Zoral Models Tuning

Tuning process - MFS models for new digital lending products

Minimum Viable Product (MVP) methodology

Zoral Analytical Data Workbench

Zoral software and tools used

Data transfer

A comprehensive set of data domains was provided by the client and loaded into Zoral ADW. Data extraction, cleansing, integration, fuzzy matching, enrichment, feature engineering and ZML model tuning were performed by the Zoral MFS Data Science and Risk Management team, based on an iterative MVP methodology as shown below,

Zoral iterative MVP methodology

Zoral iterative MVP methodology

Data sharing

Data sharing considerations included,

A range of data domains were used for decisioning, robo-underwriting workflow development and initial, ZML MFS model tuning. Data volumes included,

Data discovery and preprocessing

During predictive data discovery, raw data was uploaded into Zoral’s analytics environment. Numerous, Zoral AI/ML tools were used, including, enrichment/data transformation queries, data integration scripts and advanced data visualization techniques etc. These were applied to understand the data, detect data issues and support the feature engineering and ZML MFS model tuning process. Additional data sets were uploaded to assist with data queries/issues. A range of Zoral data discovery and data mining tools and methodologies were used to understand MNO MFS business processes, calculate MFS business KPIs and discover data additional fields that could be used as inputs for the ZML MFS models tuning phase.

The feature engineering process was started after data discovery, data quality assessment and data cleansing. Features were aggregated to customer level using numerical features, categorical features, timestamps, strings, etc. The model tuning phase was started once clean, complete data sets were obtained with predictive features and target variables.

ZML MFS Model tuning

Prior to MVP implementation, the client’s MFS platform had not been used for lending products. In absence of actual data on MNO MFS loan performance, Zoral used the following approach,

Three primary ZML models were tuned and transferred using available MNO data:

  1. Airtime credit (bills delinquency) model - predicting bill payment delinquency, (a “proxy” target,) for postpaid subscribers based on their MNO data usage - tuned and tested on MNO data.
  2. GSM Usage model - predicting customer default events for a live loan product based on subscriber MNO data.
  3. Transaction model - predicting customer default event for a live loan product based on mobile wallet financial transactions.

Models 2 and 3 were tuned, tested and compared to a number of ZML emerging market models and data. Features were mapped to MNO’s datasets and ZML models were transferred and adapted from similar markets.

After the models were tuned, they were blended into a single credit score to obtain a more robust and stable prediction of subscriber’s probability of default for digital, microloan products.

This ZML credit score achieved good predictive result, sufficient to be taken to the next stage for initial pilot use and tuning.

The credit default prediction model was tuned using client and other emerging market data.

The diagram below shows the results of the tuned ZML model. This was predictive for credit default events for all three geographies, thus providing a cross geography credit scorecard.

Cross-geography score vs initial feedback tuning, by scorebands, Emerging markets

Transferring MFS default prediction models from similar emerging markets to target MNO

A further round of model retuning was conducted in order to obtain an optimal, blended model, leveraging data from multiple similar geographies.

Such an approach made it possible to have viable underwriting platform to support a pilot micro-lending product within a short period of time. The ZML MFS models demonstrated good predictive power, automatically rejecting high-default and high fraud segment loans.

Customer revenue by blended score

Blended score vs. customer monthly revenue

Tuned ZML MFS credit model performance - area under the ROC curve (AUC) metrics

(0.5 AUC = bad (or random coin flip) prediction; 0.6 AUC = acceptable performance; 0.7 = good performance; 0.8 AUC and higher = excellent performance)

The pilot results showed, that the airtime credit model performance was strong and stable. GSM and Transaction model had a slightly lower performance, as these models were transferred from other geographies and not all features were mapped to MNO data, (NB some features were missing from the initial MNO datasets provided, so were excluded from the models).

The Airtime model was tuned with a “proxy” target. The two other models were ensembled and produced a stable, robust model for default event prediction for the MVP phase.

After applying each model to MNO customer data, Zoral divided the MNO customers into three risk bins: High Risk, Medium Risk and Low Risk.

The graphs show performance of each model for the airtime credit “proxy” target.

The three models were blended into a single, combined credit score to provide robust default prediction, credit limit estimation and customer segmentation capabilities.

The blended score was compared with various business KPIs (e.g. revenue split) to conduct a “sanity check” of the customer default prediction score. An example of such a check is described below.

Customers were segmented by blended score and divided into 10 equal parts (bins). The 1st bin represents customer with the lowest score, i.e. high risk, high default level customers. The 10th bin represents customer with highest score, i.e. low risk customers. Diagram 3 shows correlation of customer risk level with customer’s monthly revenue – the higher score, the higher monthly revenue.

If we drill down revenue structure we can see the combined score model still produces a good, predictive curve of score distribution vs. services revenue.

Revenue structure by blended score

As well as credit default, the following models were also tuned using the MNO data,

The diagram below shows the result of the transaction anomaly detection model. Each dot on the graph below represents the customer’s aggregated mobile wallet usage, the distance between dots shows customer transaction behavior similarity, i.e. similar customers are shown close to each other on the plot. Red dots show clusters of anomalous customer behavior that can be defined as fraud suspicious.

Transactions anomaly clusters

Mobile wallet transactions anomaly clusters

Decisioning Workflow implementation

Sample Digital Consumer Lending MVP workflow, ZDE

Sample Digital Consumer Lending MVP workflow, ZDE

Following the data absorption and model tuning, Zoral implemented the combined ZDE and analytics toolset as a running, MVP environment. This provided a scalable solution architecture that would allow rapid implementation and ease of customization for full scale production.

Zoral also worked with the MNO team to help generate MFS product definitions and functional system requirements.

Technical assistance was provided to help define, agree and implemented the integration communication protocols (APIs) between the MFS solution components.

ZDE was an essential part of the MFS solution. Using ZDE, MFS decisioning workflows were rapidly created, tuned and credit model and credit decision outputs integrated. A pilot MFS loan product decisioning workflow was developed as part of MVP. The three tuned models, blended score, fraud list, credit limit and business rules were implemented as part of the ZDE workflow. The diagram below shows a sample decision workflow with ZML MFS models utilized in the ZDE.

Prior to MVP launch, Zoral produced back-end prototypes and developed a demonstration App of the MVP from the customer point of view (front-end). Given the low, smart phone penetration in the market, the requirement was to demonstrate this using USSD. The diagram below shows the illustrative customer journey for MVP using USSD capabilities.

MVP Customer journey, loan acceptance

MVP Customer journey, loan acceptance

Brief summary of the mobile application

On Screen 1, a customer enters the USSD code of a loan product (the digital loan product can be advertised via SMS/MNO website/multi-channel marketing campaigns/etc.) and requests information about his potential loan.

The customer is asked to enter his national insurance number (or equivalent) at the time of requesting a loan.

The Zoral Platform, (ZDE plus the models described above), then determines whether a customer is eligible for a loan and calculates the optimal loan amount, based on a set of customer prequalified loans, (scored in batch mode), using customer MSISDN (mobile account number or other reference). On Screen 2, the customer is shown the output of the decisioning system with the eligible loan amount. A customer can accept this amount, enter a lower or a higher amount. If a customer enters a higher amount, the customer’s loan limit will be checked and a loan issued, provided the requested amount is lower than the maximum, dynamically calculated limit.

On Screen 3, the customer enters their MPIN to indicate their consent. The appropriate error messages is shown for any un-authorized action or unfulfilled user request.

If the customer accepts the loan terms and conditions, the loan amount is disbursed to the customer mobile account using the Zoral Platform. A corresponding record is created in the Zoral LMS including customer details, loan details, loan schedule, decisioning data, verification data, credit scores, model scores, limits utilization, etc.

A prequalified customer/loan sample, processed in batch mode, was used for the MVP. Real-time loan decisioning was implemented using the Zoral Platform. In addition to real-time functionality, further integration with the mobile/web/chat application was built to enable the MFS digital loan product to be provided across all MNO customer’s portfolio and channels.

The system was integrated with smartphones, feature phones, chat, USSD and web applications.

Results summary

Additional recommendations made

  1. increasing predictive data capture and range of data, (e.g. enriching data with new sources such as social network data, additional behavioral data, etc.)
  2. increasing the range of fraud models
  3. improving collections infrastructure to optimize payments from non-performing loans.

This document is provided by Zoral Limited and its affiliated companies (“Zoral”) for informational purposes only, without representation or warranty of any kind. Zoral shall not be liable for errors or omissions with respect the information contained in this document. Product Specifications are subject to change without notice. The only warranties for Zoral products and services are those that are set forth in the express warranty statements in Zoral’s standard contracts for such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

© Zoral Limited 2017