- AUC 0.78–0.91 strong predictive results for mobile lending
- 3 data sources mobile usage (GSM), airtime credit and transaction data, in emerging markets
“Is mobile-operator data predictive? Yes.”
Can a mobile network operator’s data, on its own, predict customer behaviour, credit risk, default and fraud? Is it enough to do so reliably?
This case study shows that it is. Mobile-operator data can be used on its own where bureau and banking data are thin or missing. The results were accurate and stable enough to support a mobile lending pilot. Where mobile-operator data is used for credit, it needs a legal basis and, where required, the customer’s consent, market by market.
Project summary
Data
- Collect and analyse the operator’s data: three main sources (mobile usage, network transactions and airtime credit) over 12 months.
- Clean, transform and enrich the data, and load it into Zoral Data Fabric.
Models
- Combine and tune existing credit and other models from the Zoral model library.
- Use models and data from comparable emerging markets as a proxy.
- Run the work on the Zoral Finance Operating System (fOS).
- Blend the scores of several credit models into one optimised credit, limit and fraud score.
Pilot
- Put the decisions into Zoral Decision Engine.
- Benchmark against comparable industry performance.
- Run mobile lending decisions in pilot mode, and review the results.
- Model and test a first product, based on comparable pilots.
Project goals
The main goal was to build and benchmark a pilot automated consumer lending platform, connected to the operator’s mobile financial services:
- select and score customers from the operator’s base;
- offer a first lending product to a subset of selected, scored customers;
- tune the models on the pilot’s results;
- set out a roadmap and platform for the pilot;
- show the credit performance and viability of the pilot.
Method
- The data was loaded into Zoral’s analytics environment.
- It was compared with the model library’s data templates for mobile credit. Its quality was measured with Zoral’s 4I method: invalid, incomplete, inconsistent and inaccurate.
- Where quality was poor, the data was cleaned and enriched, and other data was added to fill the gaps.
- Zoral’s existing emerging-market models were used, with Zoral Decision Engine, the Zoral model library and Zoral Data Fabric.
- Discovery, quality checks and enrichment were repeated until the data was good enough to predict.
- Business measures and targets were defined: default rate, fraud rate, retention, lifetime value and customer limits.
- The operator had no lending product in production yet, so Zoral used proxy targets from emerging markets to estimate underwriting and portfolio results.
Tuning process — MFS models for new digital lending products
- Raw data
- Data discovery
- Data quality (4Is)
- Data model template
Data path
Feature path
- Machine learning
- Models
- Deployment
Feedback → back to data discovery
Zoral Data Fabric — big data and analytics environment
Channels
Frontend services
Raw data storage
Processing
Indexes
Analytics tools
Data
The operator provided a broad set of data, loaded into Zoral Data Fabric. Zoral’s mobile data science and risk team then did the extraction, cleaning, matching, enrichment, feature building and model tuning, step by step.
Zoral iterative MVP methodology
Iterative MVP loop
- 1Business and operational questionsBusiness users bring data sets, Zoral AI/ML tools, data mining and target KPIs
- 2Value? YesA pilot leads to MVP implementation
- 3Verify MVP resultsResults are checked against the target KPIs
- 4Iterate and adoptRepeat the POC with more value-enabling initiatives and new predictive datasets, and put successful, high-value MVP initiatives into production
Analyze
Internal and external predictive data
Predictive data and model governance
- Enable real-time decisioning
- Optimize KPIs
- Data mine and enrich
- Discovery and cleansing
- Integration
- Data collection
What shaped the data shared:
- Law. Some data had to be masked.
- Access. Some data was not available for the pilot.
- Time. The pilot’s timescale limited how much data could be extracted. The pilot used only the most predictive data.
Data volumes:
- 39m prepaid mobile subscribers, with a year of monthly and daily totals
- 750,000 postpaid subscribers, with six months of monthly and daily totals
- 1.5m mobile-wallet customers, with six months of detailed transactions and daily totals (1m active customers, 15m transactions a month)
- 4 months of credit payment data
Discovery and preparation
The raw data was loaded into Zoral’s analytics environment. Zoral’s tools were used to understand it, find problems and prepare features for the models. More data was loaded where questions came up. Zoral’s discovery methods also helped calculate the operator’s business measures and find further fields for the models.
Features were built after the data was checked and cleaned. They were combined at customer level from numbers, categories, dates and text. Model tuning started once complete, clean data sets with predictive features and targets were ready.
Model tuning
The operator’s platform had not been used for lending before, so there was no loan history. Zoral therefore:
- used existing mobile credit models from similar markets;
- tuned them to proxy targets;
- compared and retuned the results;
- compared the pilot’s outputs with the same models in similar markets.
Three models were tuned and transferred using the operator’s data:
- Airtime credit model. Predicts late payment of airtime credit, used as a proxy target, from mobile usage. Tuned and tested on the operator’s data.
- Mobile usage model. Predicts default on a live loan product from the subscriber’s mobile data.
- Transaction model. Predicts default on a live loan product from mobile-wallet transactions.
Models 2 and 3 were tuned, tested and compared with emerging-market models and data. Their features were mapped to the operator’s data, and the models were transferred and adapted from similar markets.
The tuned models were then blended into one credit score, for a more robust and stable prediction of default on micro-loans. The score was predictive in all three countries tested, giving one scorecard across countries.
Cross-geography score vs initial feedback tuning, by score bands
A further round of tuning, with data from several similar countries, produced the best blended model. This gave a workable underwriting platform for a pilot micro-loan product quickly. The models showed good predictive power, and declined the segments with high default and fraud automatically.
Customer revenue by blended score bins
Model performance (AUC)
AUC measures how well a model ranks good and bad customers: 0.5 is no better than a coin toss, 0.6 is acceptable, 0.7 good, and 0.8 or more excellent.
- Airtime credit model: 0.91
- Mobile usage model: 0.82
- Transaction model: 0.78
The airtime credit model was strong and stable. The other two scored a little lower, because they were transferred from other countries and some features were missing from the first data sets. Those features were left out.
Tuned credit models — area under the ROC curve
Customers were divided into high, medium and low risk. The three models were blended into one score for default prediction, credit limits and segmentation.
As a check, the blended score was compared with the operator’s business measures. Customers were split into ten equal bands by score, from highest risk (band 1) to lowest risk (band 10). The higher the score, the higher the customer’s monthly revenue, and the pattern held across the revenue structure.
Revenue structure by blended score
Two more models were tuned on the operator’s data:
- Credit limit, based on the blended score.
- Fraud prevention, with blacklist checks and an anomaly model based on wallet transactions.
The anomaly plot shows each customer’s wallet use as a dot. Similar customers sit close together. Red dots show clusters of unusual behaviour that may be fraud.
Mobile wallet transaction anomaly clusters
The decision workflow
After the models were tuned, Zoral set up Zoral Decision Engine and the analytics tools as a working pilot environment, ready to scale to full production. Zoral also helped the operator’s team define the product and system requirements, and agree the interfaces between the components.
In Zoral Decision Engine, the pilot’s decision workflow combined the three tuned models, the blended score, the fraud list, the credit limit and the business rules.
The customer journey
Before launch, Zoral built a demonstration of the customer’s experience. Few customers had smartphones, so the pilot used USSD, the menu system on basic phones.
- The customer dials the loan product’s USSD code, advertised by SMS, the operator’s website or other campaigns. They enter their national ID number.
- Zoral Decision Engine, with the models above, checks eligibility and works out the best loan amount. It uses pre-scored offers, calculated in batch and linked to the customer’s mobile number. The customer can accept the amount, or ask for a lower or higher one. A higher amount is checked against the customer’s limit, which is recalculated each time.
- The customer enters their mobile PIN to confirm consent. Clear messages explain any request that cannot be met.
If the customer accepts the terms, the loan is paid into their mobile account. Zoral fOS passes the loan to the operator’s own core platform, where it is recorded with the customer and loan details, schedule, decision data, checks, scores and limit use.
The pilot used a batch of pre-scored customers. Real-time decisions also ran in Zoral fOS. The system was connected to smartphones, basic phones, chat, USSD and the web, so the product could be offered across the operator’s whole customer base.
Results
- Zoral fOS and the analytics tools were set up to capture and enrich the operator’s data.
- Data-quality checks found problems in the operator’s data, which were fixed.
- The pilot showed that the operator’s data, once cleaned and prepared, could support automated lending and new products.
- A production-ready default score was built by blending three high-performing models (AUC 0.78–0.91), tuned, tested and transferred from similar emerging markets.
- A credit-limit model and a fraud-prevention workflow were built in Zoral Decision Engine.
- The blended score was production-strength and closely matched the operator’s business measures, such as monthly revenue per customer.
- A production-strength credit decision workflow was built on the tuned models and the operator’s data.
- The digital lending pilot went live on the operator’s mobile financial services platform.
Further recommendations
- Capture more predictive data from the operator’s own services.
- Add more fraud models.
- Improve collections, to recover more from loans in arrears.
+44 20 3963 2002