sydney december 11, 2006 seite 1 lessons from implementations of basel ii and for solvency ii -...

Sydney December 11, 2006 Seite 1

Lessons from implementations of Basel II

and for Solvency II

-

Credit Rating Models for the Banking Book of

Banks

Sydney December 11, 2006 | 18.04.23

Seite 2

Credit Risk

• Credit risk is key for the business model of a universal

bank

• Hence, for core credit segments (retail, corporates,

banks,…) rating models were established long before

Basel II

• Rating systems actually in place were not implemented

from scratch

• Typically, they are a hybrid models blending the existing

ones with newer approaches (external data, KMV,

RiskCalc, statistical models)

Sydney December 11, 2006 | 18.04.23

Seite 3

What‘s new: Definition of Default

Institute’s View:

• Definition of Default according to the Rating Agencies and according to Basel II are almost identical

• Argumentation:

• Similar semantic definition

• Analysis of internally observed defaults delivers no statistical evidence of underestimating the PD (binomial test based on a sample containing 14 defaults) indirect argument that definitions are similar

Sydney December 11, 2006 | 18.04.23

Seite 4

Definition of Default

BaFin‘s View:

• Definition of Default according to the Rating Agencies and according to Basel II are different.

• Argumentation:

• Compared to banks, rating agencies are not able to observe all criteria belonging to the Basel II definition of default (asymmetric information)

• There even exist differences between the default definition of Rating agencies, e.g. Moody´s refers primarily to rated bonds rather than to other liabilities as for example bank loans

Sydney December 11, 2006 | 18.04.23

Seite 5

Definition of Default

Analysis of the validation data:

• 400 datasets carry a default flag

• 53 of these include an external rating

• from these 53 the external rating reflects a default state in only 14 cases

The ratio 53/14 is an indication that there are differences between the default definitions

Sydney December 11, 2006 | 18.04.23

Seite 6

Definition of Default

However, the ratio 53/14 overestimates the effect:

• Rating agencies may react after the institute has observed a default (time delay)

• Credit officer does not necessarily update the information about the external rating for internally defaulted obligors

Further analysis performed by the institute suggests a scaling factor of about 1.2 between internal and external default rates for this sample.

Sydney December 11, 2006 | 18.04.23

Seite 7

Case Study: Module Corporates

1. initial situation model developing process (MDP)

2. design of rating system „Corporates“2.1. pooling standards2.2. quantitative part2.3. qualitative part2.4. creditworthiness rating 2.5. support / burden and transfer stop

3. validation

Sydney December 11, 2006 | 18.04.23

Seite 8

1. Initial Situation MDP

basic proceeding

• pool project

• data used: quantitative ratios out of annual balance sheet and qualitative ratios (questionnaires), default information provided

• data transformation on risk points between 0 and 100. Higher value means higher risk.

• determinating weights by means of which these risk points are included in the total score (using logistical regressions and adjustment of experts)

• estimation of PD allocated to a score with logistical regression

• classifying of these individual PD in a master scale

Sydney December 11, 2006 | 18.04.23

Seite 9

data base for model development and validation

Sydney December 11, 2006 | 18.04.23

Seite 10

1. initial situation MDP

poor data quality of ratios

• ratios out of annual balance sheet are characterized by numerous and extreme outliers

• in approx. 30% of all observations at least one ratio is outside of the 1% or 99% quantile

• ratios of the qualitative section are in some cases significantly beyond the respective range

- examples are given on the subsequent pages

Sydney December 11, 2006 | 18.04.23

Seite 11

Bagplot of Balance sheet data

Sydney December 11, 2006 | 18.04.23

Seite 12

Equity capital rate 0,5% to 99,5% quantile

Sydney December 11, 2006 | 18.04.23

Seite 13

• fixing five parameters (0,25,50,75,100) and the ranges of value allocated to these five parameters

• generation of clusters depending on regions and sectors

• Clustering has a strong impact on model developing processes

• Clustering is based on profound expert know-how (e.g. external consultancy)

• especially for foreign clusters: external experts

regular check of clustering required

Transformation of the quantitative ratios in risk

points

Sydney December 11, 2006 | 18.04.23

Seite 14

equity capital rate according to clustering

high absolute frequency

with 100 risk points for

non-defaulted borrowers

Sydney December 11, 2006 | 18.04.23

Seite 15

2.1. Pooling Standards

1. population

• switching to gross and net liability according to economic point of view

• method of pool partner is unknown

2. completeness of data set

• different definitions of input box of pool partners (optional or compulsory entry) can result in different filling rate of pool input.

• example: key figure „short-range supplier credit target“

obliging guidelines for an agreement on a consistent proceeding for all pool partners are meaningful

Sydney December 11, 2006 | 18.04.23

Seite 16

Analyses by BaFin

• reconstruction of modeling and score computation on basis of the sample used for the development

• Given the data and the model as described in the documentation the error was about 100%

• Analog model development and score computation using own estimation of parameters of logistical regression maintaining data transformation (risk points and according limits)

• analysis of impact on allocation of borrowers in rating grades and estimation of PD.

Sydney December 11, 2006 | 18.04.23

Seite 17

Re-traceability of calculations

• estimation of parameters could be traced back by means of documentations and subsequent questioning (relative deviation under 0,1%)

• estimation of parameters for the quantitative ratios are sensitive with regard to different treatment of missing values (relative deviation of more than 20% using the substitution method applied for validation)

• estimation of parameters for the qualitative ratios are sensitive with regard to outliers, especially beyond the interval [0,100] (relative deviation of more than 15% for significant parameters, more than 50% for less significant ones)

• influence of individual extreme outliers on the coefficients used for the estimation of PD: 1,5% on the intercept, 2,5% on the slope (3544 observations, relative deviation)

Sydney December 11, 2006 | 18.04.23

Seite 18

comparison bank’s Model with the purely

statistical model

Sydney December 11, 2006 | 18.04.23

Seite 19

Difference of rating grades

impacts on total borrowers in-sample:

Expert-driven model assigns worser rating grades

impacts on defaulted borrowers:

Expert-driven model assigns too optimistic rating grades

Sydney December 11, 2006 | 18.04.23

Seite 20

Comparison of discriminatory power

variations of discriminatory power can be mainly observed in the lower areas for bad borrowers

Sydney December 11, 2006 | 18.04.23

Seite 21

estimation of PD

impacts on the determined PD

• estimation of parameter with logistical regression yields different results different functional relation between score and PD: expert-driven model more conservative für low scores (good borrowers), to progressive for higher scores (bad borrowers)

• different distribution of scores different distribution of PD

• small variation in average, but strong impact on single borrowers

Sydney December 11, 2006 | 18.04.23

Seite 22

conclusions

• Due to the high importance of qualitative ratios, quality assurance of inputs is treated with special importance.

• The influence of experience of credit experts on the different steps of modeling should be checked within validation.

• In-sample shows the expert-based model weaknesses especially with regard to the allocation of worse borrowers.

• Analog analysis should be executed out-of-sample and out-of-time .

Sydney December 11, 2006 | 18.04.23

Seite 23

Data for model development process and

validation

Sydney December 11, 2006 | 18.04.23

Seite 24

Data

Sydney December 11, 2006 | 18.04.23

Seite 25

Summary Statistics

Sydney December 11, 2006 | 18.04.23

Seite 26

B1 : Equity Capital Ratio

Sydney December 11, 2006 | 18.04.23

Seite 27

Boxplot of equity capital ratio C

Sydney December 11, 2006 | 18.04.23

Seite 28

Trimming of Variable B1

Sydney December 11, 2006 | 18.04.23

Seite 29

Boxplot of equity capital ratio Cγ

Sydney December 11, 2006 | 18.04.23

Seite 30

Sydney December 11, 2006 | 18.04.23

Seite 31

Sydney December 11, 2006 | 18.04.23

Seite 32

Estimates by QRM

Sydney December 11, 2006 | 18.04.23

Seite 33

Influence of an Outlier

Sydney December 11, 2006 | 18.04.23

Seite 34

Descriptive analysis of default probability

mean value and standard deviation

• mean value as per model: 0.95%

• standard deviation as per model: 2.09%

• mean value with expert influence: 0.96%

• standard deviation with expert influence: 1.84%

Deviations (Model - expert-driven model)

• mean value: -0.0015%

• minimum: -22.23%

• maximum: 23.54%

Sydney December 11, 2006 | 18.04.23

Seite 35

estimation of default probability

effects on the calculated default probability

• estimation of parameters by means of logistic regression is providing other results for coecients

– another function connecting score and default probability: PD curve of expert-driven model proves to be more conservative in lower score area (good borrower)and more progressive in the upper score area (bad borrowers)

• other distribution of scores

– other distribution of default probability

• high impact on individual borrowers

Sydney December 11, 2006 | 18.04.23

Seite 36

Sydney December 11, 2006 | 18.04.23

Seite 37

Sydney December 11, 2006 | 18.04.23

Seite 38

Sydney December 11, 2006 | 18.04.23

Seite 39

Sydney December 11, 2006 | 18.04.23

Seite 40

Sydney December 11, 2006 | 18.04.23

Seite 41

Sydney December 11, 2006 | 18.04.23

Seite 42

Sydney December 11, 2006 | 18.04.23

Seite 43

Sydney December 11, 2006 | 18.04.23

Seite 44

Sydney December 11, 2006 | 18.04.23

Seite 45

Sydney December 11, 2006 | 18.04.23

Seite 46

Sydney December 11, 2006 | 18.04.23

Seite 47

analysis of defaults

Sydney December 11, 2006 | 18.04.23

Seite 48

influence of expertise on the model

The expertise of credit department has a vital influence on the model building process:

• following the existing model → model selection

• selection of the analysed variables → selection of variables

• determination of cluster and class limits for the allocation of risk points

→ data transformation

• determination of weights of quantitative variables

• determination of weights of quantitative partial score and the qualitative variables

→ determination of score function

• definition of qualitative variables, evaluation of qualitative variables

→ subjective evaluation

Sydney December 11, 2006 | 18.04.23

Seite 49

Analysis of the resulting effects

• tracing back the model building process and score computation on basis of the data set submitted to the subvisors

• analog model building process and score computation using the parameter estimation of the logistic regression and maintaining the risk points and class limits)

• analysis of eects of the assignment of rating classes to borrowers and the estimation of default probability

Sydney December 11, 2006 | 18.04.23

Seite 50

Some conclusions

• Due to the high importance of the qualitative variables and the sensitivity of parameter estimation concerning outliers, quality assurance of input is attached special importance.

• The influence of expertise of credit departments on the dierent steps of modeling should be checked within the validation process.

• In-Sample shows the expert-driven model weaknesses especially with regard to allocation of bad borrowers.

• analogue analysis should be checked out-of-sample and out-of-time.