Using MATLAB to Move to the Next Generation of GRADE Model
Nadége Lespagnol, Euler Hermes
Euler Hermes (EH) is the leading B2B credit risk business of the Allianz Group, helping customers protect themselves from bad debt.
EH has a strategic objective to centralize all credit assessment model calibration data, model design, and model monitoring processes in one common modeling platform, helping to meet the regulatory requirement for reconciliation and transparency for all credit assessment models. EH’s proprietary GRADE model is a probability of default (PD) model used in both the underwriting process and in the allocation of risk capital.
In 2020, EH launched a transformation project to migrate all credit risk models from a legacy infrastructure to MATLAB® running on AWS®. EH used components of the MATLAB Model Risk Management solution to develop and maintain the full suite of credit risk models, which are based on fuzzy logic approaches as well as tree-based algorithms.
In this presentation, learn how EH has built a new model design architecture with MATLAB and AWS that will allow many improvements in the process of building and testing future models.
Published: 6 Oct 2021
Hi, everybody. I'm Nadége Lespagnol. I'm the group head of Credit Assessment Models at Euler Hermes. And I'm going to talk to you about a project we've been partnering with MathWorks around the future of our grading models.
So I've got personally over 20 years experience in modeling for financial industry. These are banks and insurance companies. And we've been experimenting over these past years different techniques of modeling since techniques have evolved ethically amongst all of those years. And we really want it to be at the state of the art in terms of usage of models for great.
I've got a team of data scientists that are engineering and the credit experts. And in early 2020, we decided to launch the new grading platform project. And we selected MATLAB as a software and as a platform amongst 13 provider in order to start off project end of last year.
OK, so in terms of business model, what do we do at Euler Hermes? First of all, we are a credit insurer. So this means that we are insuring companies against trade credit insurance issues from other companies they are doing trade trading with. So at Euler Hermes, we are working on risk monitoring and debt collection in order to secure this business. And to help enable this business, we need models to make the best decisions.
Well, we have 55,000 customers around 60 countries. We have access to 350 million companies across the world in terms of data. And we have $950 billion exposure with a number of credit limit requests of 20,000 per day. So lots of activities that require lots of automation.
In order to support this business, therefore, we need to assess the financial risk. And we use models to assess this risk with a scale of 1 to 10 from exceptional to failure.
So how information is translated into decision? So first of all, we have to have enough data to build proper models and make proper decision. So we collect data through data providers, many of them. We also collect data through our credit analysts, our policyholders, collection agencies, open data sets, and many other sources like bonds, for example.
Then we process this data in order to put them in the right format, in the right structure in order to monitor data quality, and then apply the rules in order to pre-process the information to make this information usable from a modeling standpoint. To build our assessment, we are assessing risk on internal rules and automated or manual company level risk assessment.
The automated assessment is based on a Fuzzy Logic approach model. And we have 60 models across the world based on this Fuzzy Logic approach. But we also use machine learning on some core countries in order to challenge these models and make sure that we have the best decisions on our larger exposure countries.
In terms of application, for sure, this grade, this EH Grade is used for decision on credit limits in trade credit insurance business across the world. So again, 60 countries with 60 different models, and 60 different decision systems adapted to local specificities.
So in order to support this business, for sure, we need a robust platform. So before early 2020, the process was fairly robust, but very manual-intensive spread across different technologies. And we wanted to put everything in one single platform in the cloud. And to support this decision science platform, we wanted to select the best software to enable this.
So first of all, we wanted to get the ability to deal with many data sources since we have a large type of data with a very large database to manipulate and work with. Again, the data prepossessing is a key part of our work. Many hours are spent to ensure the best data quality, the best understanding of our data.
And we are spending also some time on understanding the data bias. Produces bias produce the issues in terms of quality. We are also considering advanced segmentation techniques. And for sure, sampling is at the heart of all of our process.
But where we really rely on the technology, and probably where we found MATLAB very useful is a capacity to cover any type of algorithm. As mentioned, Fuzzy Logic is our incumbent approach. However, this approach we used was very much expert-based at the beginning. And we wanted to bring more optimization techniques in this process.
On top of that, we have a machine learning process as a challenger. We need to maintain and keep improving to ensure that we are getting the best insight out of this process to feed our incumbent approach. And therefore, we are also considering hybrid models in research and development mode for the moment in order to bring these two approach together.
We also have to invest on model explainability since we are using AI. Model validation is also, of course, a process of model design environment as well as modern monitoring. And finally, we also wanted to integrate this platform into our production system. So the capacity to deploy our model in a very, I would say, in the industrial mode was a key requirement when we decided to move on with this project.
And finally, the model lifecycle management is a core piece of our process. And MATLAB is also offering some great functionality to build up on this part. So in the conclusion, we want transparency and auditability of our end-to-end model design and monitoring process. And that's why we decided to invest in this large transformation project for our team.
One enabler when choosing MATLAB was the capacity to rely on the existing Fuzzy Logic toolbox. Fuzzy Logic is an approach that is very different from what is used generally in the financial services industry. It's enabling notably to model non linearity, however, in a very explainable way, which is why we decided to use Fuzzy Logic. It was in 2010, so more than 10 years ago.
And the Fuzzy Logic models we are currently using are consider that type-1 Fuzzy Logic approach. And we are willing to move to type-2 for the inference systems that are more optimized and more statistically-driven, shall I say, to enable better modeling and experience, and to make sure that we can optimize further accurate models. So that's why we decided to acquire this Fuzzy Logic toolbox. That is including already a lot of different options to enable these type of models.
The use of Fuzzy Logic is generally used in different industries like consumer electronics or automotive, for example. Less in the financial services, but still very robust. And again, it's enabling non linearity modeling. It's enabling us to model or to mimic the credit analyst way of thinking when making an assessment or risk assessment. And we would like really to keep working with this approach that is quite good in terms of performance and, again, explainable.
So what we've been doing these past few months with the support of MATLAB consulting teams is a migration from the legacy system to the MATLAB platform we built in the cloud. So we've been building automated import of models from this legacy system we used to use into the Fuzzy Logic toolbox for MATLAB.
And to do that, we used a piece of code that was enabling this migration in a much more industrialized way since we have 16 models to migrate. It's quite a huge number of models. To do so, we had to move to use an automatic way.
We also have been working on the customized model environment, model development environment, including some graphical debugger to enable the team to visualize and facilitate the debugging of our models, which was great, with the support of MATLAB, was the fact that they help us to customize this part, which was not necessarily already built-in in the existing MATLAB toolbox. So we've been working on this development together. And that was a good experience for us to be able to customize our content.
So as a result, we converted 60 models automatically. All models are reproducing the results of the legacy system. What we found very good in terms of added value also is the fact that MATLAB calculation software was more precise. We had a higher resolution in the membership functions of our models and the conversions therefore allowed for systematic cleanup of all models.
In terms of the future, we are considering type-2 Fuzzy Logic approach. Why? Because many papers have been published on the fact that this approach was considered as very robust and performing and explainable one from a regulatory standpoint. And the financial services are currently considering this as a potential contender compared to usual logistic linear regressions or even machine learning approaches.
So that's why we decided to launch an experimentation using the MATLAB toolbox Fuzzy Logic toolbox in order to demonstrate what will be the value added if we move to this new type-2 Fuzzy Logic model. So, so far, we are still in the experimentation mode. Research and development project is ongoing. And we believe that there will be added value in moving in this kind of framework moving forward.
We have great expectations to launch this project in a very operational way starting at the end of the year. So that's why we are pushing a lot on-- and we are putting a lot of efforts on this project as I speak.
What we are also considering is introducing machine learning. Type-2 Fuzzy Logic is also considered that AI. So that's why we are also investigating other approach called hybrid approach where we are considering a mixture of traditional models, meaning some linear model with feature and advanced feature engineering, and had some learnings from what we can get out of AI or machine learning type of models.
So that's why we have to invest in developing a model development environment that is enabling all of this type of approach not only Fuzzy Logic, but also hybrid models and mixed together, developing an industrialized platform for the team to work in in order to ensure transparency and whole business for the model as mentioned.
But whole producibility or so of our models is also key since they are regulated models. We need to ensure this whole producibility. And we will experiment as the experiment manager piece of MATLAB with that purpose. We are also considering the automation of our documentation and reporting using some MATLAB functionality and apps allowing this.
We've been supported across all time by the MathWorks team. So the team has been trained on different techniques, modeling techniques like machine learning, for example, but not only. And we have constant coaching for our software development, meaning our model development environment, but also to ensure proper deployment of our model from a technical standpoint. So we have a team of experts with values, background data, engineering modeling, but also capacity to integrate in the cloud, which is making on its own quite a big project with the different actors and stakeholders.
The next steps are introducing the model development environment into production that will be for early 2022 or maybe end of year if it goes as expected. And for sure, the implementation of alternative models is very much at the heart of our expectation. And that's a project that is followed by your board of management on a regular basis.
That's it for my presentation. So I'm pleased to hear any question you may have. Thank you very much.