alternative data

Blog | Raising the Stakes on Psychometric Credit Scoring

An updated and expanded 2nd edition (first edition)

Why read this post?

Learn why high-stakes data is essential for building accurate credit-scoring models.

 

Introduction

Billions of people lack traditional credit histories, but every single person on the planet has attitudes, beliefs, and behaviors that can be used to predict creditworthiness. Quantifying these human traits is the focus of psychometrics, and the alternative data provided by this technique allows LenddoEFL to greatly expand financial inclusion in its mission to #include1billion.

But there is a catch: in order to build models that accurately predict default, applicants need to complete psychometric assessments in pursuit of actual financial products, a so-called “high-stakes” environment. This is because people answer psychometric questions differently when they have a chance to receive a loan (the high stakes) than they would in a hypothetical situation with no incentive (the low stakes).

Despite this fact, psychometric tools are frequently built using low-stakes data. For example, many companies develop psychometric credit scoring tools using volunteers. And many lenders want to validate psychometric credit scoring tools on their clients through back-testing: giving the application to existing clients and comparing scores to their repayment history, again a low-stakes setting.


These approaches are only valid if low-stakes data can be applied to the real world of high-stakes implementation, where access to finance is on the line for applicants. But it turns out that this is not the case. A recent study published by our co-founder Bailey Klinger and academic researchers proved that low-stakes testing has no predictive validity for building and validating psychometric credit scoring models in a real-world, high-stakes situation. The data below shows exactly how applicant responses shift as they move from one environment to another.

 

The Experiment

To test for differences between low- and high-stakes situations, LenddoEFL gathered psychometric data from two sets of micro-enterprise owners in the same east-African country. One group already had their loans (low-stakes) and another group completed a psychometric assessment as a part of the loan application process (high-stakes).

First, the low-stakes data. The figure below shows the frequency distribution for two of the most important ‘Big 5’ personality dimensions for entrepreneurs, Extraversion and Conscientiousness, as well as a leading integrity assessment[i].
 

image1.png


You can see that when the stakes are high, people are answering the same questions very differently. The distribution of scores on these three personality measures shifts significantly to the right. When something important is at stake, like being accepted or rejected for a loan, people answer differently.

How do these differences in low- vs. high-stakes data matter for credit scoring?

To see how these differences impact the predictive value of psychometric credit scoring, we can make two models[ii] to predict default: one uses responses from applicants that took the application in low stakes settings, and the other uses responses from applicants that were in high stakes settings. Then we can use a Gini Coefficient—which measures the ability of a model to successfully rank-order applicants’ riskiness and for which a higher coefficient is a metric of success in this—to compare each model’s ability to predict default for the opposing population as well as its own.[iii]

image2.png


These results clearly show that there is a significant change in the rank ordering when models built on low-stakes data are applied in high-stakes settings and vice versa.[iv] Importantly, we can see that a psychometric credit-scoring model can indeed achieve reasonable predictive power in a real-world, high-stakes setting. But, that is only when the model was built with high-stakes data.

Think about it like this: when the stakes are high, both less and more risky applicants change their answers. But, less risky applicants change their answers in a different way than riskier applicants. This difference is what is used to predict risk in psychometric credit scoring models: the difference between how low- and high-risk people answer in a high-stakes setting.

This also illustrates why we see that a model built on low-stakes data is ineffective in a real-world high-stakes implementation. In the low-stakes setting, the low- and high-risk people aren’t trying to change their answers, because they aren’t concerned with the outcome of the test. Once the stakes are high, however, this pattern changes.

 

Conclusions

Testing existing loan clients or volunteers has an obvious attraction: speed. That way you don’t have to bother new loan applicants with additional questions, and then wait for them to either repay or default on their loans before you have the data to make or validate a score, an approach that takes years.

Unfortunately, these results clearly show that this shortcut does not work. People change their answers when the stakes are high, so a model built on low-stakes data falls apart when used in the real-world. People answer optional surveys with less attention and less strategy than they do a high-stakes application, and therefore the only strong foundation to a predictive credit-scoring model is real high-stakes application data and subsequent loan repayment.

Consider an analogy: you can’t predict who is a good driver based on how they play a driving video game, where the outcome is not important. Conversely, someone who does well on a real-world driving test may not perform that well on a video game.  Whether it is driving skills or creditworthiness, you must predict the high-stakes context with high-stakes data.

 

TAKEAWAYS:

- Psychometric model accuracy is only guaranteed when you collect data in a high-stakes situation (i.e., a real loan application).

- Despite its speed, back-testing a model on existing clients in a low-stakes setting is risky because it might not tell you anything about how the model will work in a real implementation.

- If you want to buy a model from a provider, the first thing you should verify is what kind of data they used to make their model. Was it from a real-world high-stakes implementation similar to your own?

 


[i] These are indices from widely available commercial psychometrics providers. It is important to note that LenddoEFL no longer uses any of these assessments or dimensions in our assessment, nor any index measures of personality.

[ii] Stepwise logistic regression built on a random 80% of data, and tested on the remaining 20% hold-out sample. An equivalently-sized random sample was used from the other set (high-stakes data for the low-stakes model, and low-stake data for the high-stakes model) to remove any effects of sample size on gini.

[iii] Note that this exercise was restricted to those questions that were present in both the low- and high-stakes testing. It does not represent LenddoEFL’s full set of content and level of predictive power, it is only for purposes of comparing relative predictive power.

[iv] The results also show that using standard personality items, the absolute predictive power is lower in a high-stakes setting compared to a low-stakes setting. This is likely because of the ability to manipulate some items in a high-stakes setting makes them not useful within a high-stakes setting. This lesson has lead LenddoEFL to develop a large set of application content that is more resistant to manipulation and which has much higher predictive power in high-stakes models. This content forms the backbone of the current LenddoEFL psychometric assessment, all of which is built and tested exclusively with high-stakes data and subsequent loan repayment-default rather than back-testing.

 

Forbes | Could Personality Tests One Day Replace Credit Scores?

forbes article 042018.jpg

If someone gave you an unexpected $100, what would you do with it? Give it to charity? Save it? Splurge on something fun?

We see questions like this in personality quizzes online, and sometimes even when applying for jobs. Your answers are supposed to help others predict your behavior using what’s called psychometrics.

And companies looking to avoid hiring potential problem employees aren’t the only institutions interested in psychometrics. The financial industry might get in on it, too.

What if, instead of a lender checking your credit score, they gave you a personality test?

Read full article.

Spore Magazine | Réduire les risques : Des systèmes innovants d’évaluation du crédit pour aider les agriculteurs

Screen Shot 2018-04-26 at 6.07.32 PM.png

La difficulté d’emprunter, pour de nombreux petits agriculteurs ne disposant ni de garanties ni d’antécédents de crédit, a fait apparaître de nouveaux systèmes pilotes d’évaluation du crédit pour aider les banques à apprécier les risques que présentent réellement les emprunteurs et tirer parti de ce secteur potentiellement lucratif.

L’évaluation psychométrique

Pour augmenter les taux d’acceptation et réduire les délais de traitement des prêts aux agriculteurs, Juhudi Kilimo, prestataire de solutions financières pour les petits agriculteurs d’Afrique de l’Est, teste la méthode d’EFL Global, une entreprise privée qui utilise l’évaluation psychométrique pour créer les profils de risque d’emprunteurs africains, asiatiques, européens et latino-américains. Cette méthode pilote – financée par la Fondation Mastercard – mobilise les représentants de six agences kényanes de Juhudi qui visitent et incitent les demandeurs de prêts à passer des tests psychométriques sur tablette. Ces tests permettent, selon EFL, de définir leur personnalité, y compris leur self-control en matière de dépenses et budgétisation. Sur cette base, une cote de crédit à trois caractères est alors attribuée aux demandeurs. À partir de son évaluation initiale d’environ 6 000 clients réalisée à l’aide de l’outil d’EFL, Juhudi a constaté que 6 % des personnes classées dans le quintile le plus bas avaient au moins une fois des arriérés de remboursement de 60 jours pour un prêt type d’un an, contre 1,5 % dans le quintile le mieux noté.

Read full article.

Microfinance Gateway | Malaysia: Fintech Heavyweight CTOS Expands Services for A Better Financial Inclusion

CTOS has been Malaysia’s largest in terms of credit reporting, just announced a partnership with LenddoEFL to achieve a joint vision of financial inclusion for the people who had difficulties securing loans in Malaysia due to the lack of credit history. 

Read article in MicroFinance Gateway website: https://www.microfinancegateway.org/announcement/malaysia-fintech-heavyweight-ctos-expands-services-better-financial-inclusion

Financial Express | Credit bureau veteran Darshan Shah joins LenddoEFL as Managing Director

“Having worked across geographies and being well-versed with the problem of credit coverage, I look forward to leveraging my experiences to work on the challenge of financial inclusion in India. The need is massive with less than 45% of Indian adults included in the credit bureau and less than 10% borrowing from a financial institution in the last year, as per the World Bank.” said Darshan Shah.

Read full article in Financial Express

AstroWani | CTOS, LenddoEFL extends financial inclusion in Malaysia

30% of Malaysians with good potential is still denied access to loans. This is because they lack or directly have no credit history. In order to curb this issue, Malaysia's Largest Credit Reporting agency, CTOS Data Systems Limited, partnered with Fintech LenddoEFL company and emerged with a new solution.

30% of Malaysians with good potential is still denied access to loans. This is because they lack or directly have no credit history. In order to curb this issue, Malaysia's Largest Credit Reporting agency, CTOS Data Systems Limited, partnered with Fintech LenddoEFL company and emerged with a new solution.

World Bank | Using a PhD in development economics outside of academia: interviews with Alan de Brauw and Bailey Klinger

Today's interviews are with Alan de Brauw, a Senior Research Fellow in the Markets, Trade, and Institutions Division at the International Food Policy Research Institute; and Bailey Klinger, the founder and (until recently) CEO of the Entrepreneurial Finance Lab

Read full interview with Bailey Klinger.

Media Telecom | Orange Bank comienza a ofrecer micropréstamos personales

Micropréstamos: un negocio en aumento

La posibilidad de ofrecer micropréstamos a los usuarios tienta cada vez más a la industria. No solo a la banca digital. El año pasado, Telefónica de España presentó Movistar Money. Se trata un servicio de préstamos al consumo. Asimismo, una de sus principales características es que son preconcedidos a los clientes de la operadora.

En Latinoamérica esta tendencia es todavía más importante. Así, en México, Lenddo y Entrepreneurial Finance Lab (EFL) se fusionaron para brindar productos financieros para el sector no bancarizado. Read full article.

The ASEAN Post | The potential of big data for microfinancing in Southeast Asia

"Microfinance is described by the Financial Times Lexicon as a service where financial institutions will back small start-ups and would-be entrepreneurs with small loans, in the poorest parts of the world. In Southeast Asia, the biggest microfinance players currently include Asia Pacific-based LenddoEFL, Singapore's CredoLab and the Philippines’ Lendr, for example..." Read full article.

Lodex Blog | The Future of Data-Driven Financial Inclusion Posted by Aisha Hillary-Morgan

"In Australia, millions of people find themselves in a chicken-or-egg-type dilemma when it comes to getting credit. Even though they have steady income, they still can’t access credit because of lack a formal credit history. Yet, most of these consumers carry a smartphone, are online and connected through social networks, leaving behind a digital footprint that can be analyzed to better understand who they are and their attitudes toward credit.

This is why we have teamed up with LenddoEFL, the leading technology platform powering data driven decisions for financial services, to help them create more of a credit story. Your Social Score will use, with your consent, your digital footprint to provide additional insights for borrowers and for lenders to more efficiently make a preliminary assessment." Read the full article

Originally posted by our partner Lodex

New Peer-to-Peer Lending players in Vietnam

The establishment of Lendbiz comes on the heels of the launch of LenddoEFL in Vietnam. Orient Commercial Joint Stock Bank (OCB) announced last week that it will become the first bank to offer LenddoEFL’s psychometric scoring solution in Vietnam.  Read full article.

LenddoEFL psychometric scoring solution launches in Vietnam

LenddoEFL launches in Vietnam [Finovate.com]

OCB will be the first bank to offer LenddoEFL’s psychometric scoring solution in Vietnam, a region where many citizens lack traditional financial and credit information. Using the scoring tools, OCB aims to serve more self-employed and salaried banking members. “This partnership with one of Vietnam’s leading banks marks our launch in Vietnam and part of our expansion plans as we provide fast, affordable and convenient financial products for more than 1 billion people worldwide,” said Richard Eldridge, CEO of LenddoEFL.

LenddoEFL and Orient Commercial Bank join together to serve the unbanked of Vietnam

How email and smartphone data help you get a loan

What your phone habits reveal about you

SoFi is preparing to launch in Sydney, its first market outside of the US, and earlier this year the country's first loans and deposits marketplace, Lodex, formed a partnership with Singaporean start-up Lenddo to bring its social scoring technology to the country...



Read more: http://www.afr.com/technology/how-email-and-smartphone-data-could-help-you-get-a-loan-20171212-h02zi0#ixzz534zFfQmg 

Three ways alternative data will become more mainstream in 2018

Overseas lenders can develop credit scores based on mobile and web data
FICO uses LenddoEFL’s credit scoring model overseas, which includes email, mobile and web data to assess of thin-credit file consumers in overseas markets. This is the technology behind FICO’s recently launched X Data Score in India, which generates a score based on a consumer’s mobile and digital footprint, including email data. FICO also has a traditional score in India as well.

Can behavioral traits help financial institutions assess creditworthiness?

The Problem: Financial Exclusion

Financial inclusion is a defining challenge for this generation. Many governments and supranational agencies are investing to solve this problem. Even fintech companies are trying to help, but what is the real problem and how could it be solved?

The World Bank states “Around two billion people don’t use formal financial services and more than 50 percent of adults in the poorest households are unbanked. Financial inclusion is a key enabler to reducing poverty and boosting prosperity.”

You might ask, why isn’t this population going to financial institutions to improve their living conditions, and why haven’t financial institutions served them? From a business perspective the opportunity at a global scale is massive.

Reaction from Mahindra Finance: this merger will help FIs reach out to a billion Indians for their lending needs

" Lenddo & EFL both are global pioneers in using non-traditional data, be it social, mobile or psychometric evaluation. Their coming together opens up new frontiers of collaboration with financial institutions that will help financial institutions to reach out to a billion Indians for their lending needs across customer segments."

- Tina Singh  - Chief Digital Officer, Mahindra Finance.