targeted review of internal models - europa

Targeted Review of Internal Models Project report

April 2021

Targeted Review of Internal Models – Contents

1

Contents

1 Executive summary 3

2 Purpose and scope of the report 10

3 Overview of the TRIM project 11

3.1 TRIM context and objectives 11

3.2 Scope of TRIM 13

3.3 Project roadmap and communication 17

3.4 TRIM project governance 21

3.5 Project deliverables 23

4 Observations and findings 27

4.1 General topics 27

4.2 Credit risk – models for retail and SME portfolios 32

4.3 Credit risk – models for low-default portfolios 49

4.4 Market risk 67

4.5 Counterparty credit risk 76

5 Overview of supervisory follow-up 83

5.1 Approach to evaluating the supervisory follow-up 83

5.2 Supervisory follow-up for general topics 84

5.3 Credit risk –models for retail and SME portfolios 85

5.4 Credit risk – models for low-default portfolios 89

5.5 Impact of TRIM supervisory decisions 92

5.6 Reduction of non-risk-based RWA variability through supervisory follow-up 97

6 Project review 105

6.1 Achievements and project outcomes 105

6.2 TRIM’s contribution to improving ongoing and future supervisory work on internal models in the SSM 112

7 Main achievements and conclusions 115

Targeted Review of Internal Models – Contents

2

7.1 Main achievements of the TRIM project 115

7.2 Main conclusions of the results of TRIM 118

Appendices 120

Supplementary information for Section 3.5 (Project deliverables) 120

Supplementary information for Section 5.5 (Impact of TRIM supervisory decisions) 122

Targeted Review of Internal Models – Executive summary

3

1 Executive summary

Following the financial crisis of 2007-09, concerns were raised regarding the unwarranted (i.e. non-risk-based) variability of the outputs of models used to calculate regulatory capital requirements. This coincided with concerns from banking supervisors and external stakeholders about the complexity of such models and the resulting opaqueness of the modelling approaches. This opacity also made it increasingly difficult for supervisors to assess whether risks have been captured correctly and consistently through these models.

The targeted review of internal models (TRIM) was a multi-year project launched by the ECB at the beginning of 2016 in close cooperation with the national competent authorities (NCAs) that are part of European banking supervision. TRIM aimed to assess whether the Pillar I internal models used by significant institutions (SIs)1 within the Single Supervisory Mechanism (SSM)2 are appropriate in the light of the applicable regulatory requirements and whether their results are reliable and comparable. Furthermore, TRIM aimed to harmonise supervisory practices relating to internal models within the SSM.

The TRIM project was part of a wider set of initiatives designed to address non-risk-based variability of model outputs, and complements from a supervisory standpoint the regulatory initiatives led by the Basel Committee on Banking Supervision (BCBS) and the European Banking Authority (EBA).

TRIM is the largest project so far conducted by ECB Banking Supervision in coordination with NCAs. It marked an important milestone in improving the comparability of outcomes of internal models used by SIs and increasing transparency about the ECB’s understanding of applicable regulations. In addition, it further harmonised supervisory practices by developing a common assessment methodology and enhancing collaboration across European banking supervision. In particular, the wealth of results delivered through TRIM, and the intense, detailed supervisory follow-up initiated with the institutions involved, have played, and will continue to play, a key role in promoting a level playing field and high quality standards for internal models used by SIs.

At the core of TRIM was the execution of 200 on-site internal model investigations (IMIs) across 65 SIs. The project covered internal models for credit, market and counterparty credit risk.3 Given the large number of approved internal models at SIs, and the time needed for on-site investigations, TRIM adopted a targeted approach in order to review those topics deemed to contribute most significantly to unwarranted 1 SIs are banks directly supervised by the ECB under the SSM and are the focus of TRIM. In this report,

these may be referred to as “significant institutions”, “institutions” or “SIs”. Banking institutions more generally are referred to as “banks”.

2 The SSM is the system of banking supervision established by the SSM Regulation. See Council Regulation (EU) No 1024/2013 of 15 October 2013 conferring specific tasks on the European Central Bank concerning policies relating to the prudential supervision of credit institutions (OJ L 287, 29.10.2013, p. 63).

3 Operational risk was excluded in view of the BCBS’s stance against using internal models for this risk.

TRIM complemented from a supervisory standpoint the regulatory initiatives of the BCBS and the EBA to address perceived drawbacks of internal models following the financial crisis

Under TRIM, 200 on-site model investigations for credit, market and counterparty credit risks were performed across 65 SIs

http://data.europa.eu/eli/reg/2013/1024/oj





4

risk-weighted assets (RWA) variability. The review of credit risk models focused only on those determined to be the most material and critical models. In this way, TRIM was able to cover essentially all SIs with internal models, all market and counterparty credit risk models,4 and a very significant and representative proportion of credit risk models.

Overall, the outcomes of the TRIM investigations confirmed that the internal models of SIs can continue to be used for the calculation of own funds requirements. However, for a certain number of models, limitations5 were needed to ensure a level of own funds that was appropriate to cover the underlying risk. This was notably the case for a number of loss given default (LGD) and credit conversion factor (CCF) models related to low-default portfolios (LDPs)6, for which supervisory backstops were imposed as a result of the TRIM investigations.7

Moreover, a number of deficiencies (“findings”)8 were identified that require significant effort by institutions to remediate. Although over 5,800 findings were identified across all risk types9 – a reflection of the detailed and in-depth assessment approach adopted in TRIM – it should be noted that in some cases these findings were driven by historic deviations in how requirements have been understood at national level, or by a prior lack of clarity or guidance on the implementation of certain requirements. There have also been deviations from the draft provisions stemming from the regulatory initiatives led by the EBA, such as the regulatory review of the internal ratings-based (IRB) approach to credit risk. In this respect, SIs should benefit in future from the detailed guidance provided by the ECB (in its guide to internal models) and by the relevant regulatory technical standards and EBA guidelines.

At a more aggregated level, about 30% of the findings raised in TRIM on-site investigations have a high severity, i.e. F3 or F4.10 The following main weaknesses were identified for the different types of risk.

• Credit risk models related to retail and SME portfolios: institutions generally have the capabilities to build adequate IRB models. More specifically, for the

4 Certain models, such as for correlation trading and credit valuation adjustment, were also defined as

being out of scope of the project. 5 A limitation restricts or modifies the (permitted) use of a model. For example, a restriction might prohibit

the use of the model for certain portfolios, whereas a modification might require changes to the values of certain model parameters or to the calculated own funds requirements. Where there are several addressees, and if relevant, it must be clear to which addressee an individual limitation relates. Limitations must be complied with, or the ECB may revoke the permission. Limitations must correspond to one or more findings and are linked to the fulfilment of specific obligations. The limitation will start to apply at the same time as the permission granted under the decision. See also the Guide to on-site inspections and internal model investigations on the ECB’s banking supervision website.

6 For the purpose of this report, exposures to corporates – other (with a focus on medium and large corporates), institutions, and corporates – specialised lending (excluding exposures covered by the slotting approach) are labelled as LDPs.

7 This outcome is consistent with the BCBS’s stance reflected in the final Basel III standards, which no longer permit the use of internal models for calculating the LGD of some LDPs.

8 Supervisory findings (referred to in this report as “findings”) relate to identified items requiring immediate supervisory attention to counteract deficiencies at an institution.

9 Includes findings from the general topics review. 10 The severity of a finding is based on the actual or potential impact on the institution’s financial situation,

level of own funds or own funds requirements, internal governance, risk control or management. The ECB categorises severity on a scale from F1 to F4, where F1 refers to a low impact and F4 to a very high impact.

TRIM confirmed that the internal models of SIs can continue to be used for the calculation of own funds requirements, subject to supervisory measures to ensure an appropriate level of own funds requirements at all times

More than 5,800 findings were identified across all risk types, of which about 30% have high severity (F3 or F4)

https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.osi_guide201809.en.pdf?49b4c0998c62d4ab6f31a4733c7ea518



5

probability of default (PD) parameter, more than 70% of investigations ended without severe findings on the calculation of one-year default rates and the long-run average default rate (LRA DR). However, a significant number of findings11 were raised concerning the low risk differentiation of these models, owing to the low discriminatory power of the scoring/ranking functions. Further improvements in the calibration approaches are still required, particularly as regards the need for adequate data to ensure that PD estimates reflect long-run average default rates and are sufficiently conservative. For LGD models, the calculation of realised LGD was a frequent cause of compliance issues (findings on this calculation were raised in each investigation). Moreover, 42% of investigations contained severe findings on risk differentiation. In 95% of the investigations where the LGD parameter was reviewed at least one high-severity finding (i.e. F3 or F4) was raised in relation to this parameter. For the PD parameter, at least one high-severity finding was raised in 67% of investigations.

• Credit risk models related to LDPs: a large number of findings12 were raised in relation to the rating assignment process13 and risk quantification. These mainly concerned the calibration methodology and the calculation of long-run average default rates. One of the reasons for these deficiencies is that there are considerably fewer internal observations available for this type of portfolio (when compared with retail and SME portfolios), which means that institutions have to make greater use of other observations (e.g. external default data) in order to calculate default rates and, subsequently, PDs. For the LGD parameter, most of the findings raised concerned the calculation of the realised LGD and long-run average LGD. It was observed that some institutions had difficulties in finding representative data for these portfolios, which led to cases where the LGD estimation was not based on realised LGD or representative data. As a result, there was an increased use of limitations to avoid an underestimation of capital requirements. Overall, in 96% of the investigations at least one F3 or F4 finding was raised in relation to the PD and LGD parameters.14

• All the credit risk on-site investigations included dedicated data quality reviews of IRB data. Although institutions have made efforts to ensure there are sound data management and quality frameworks in place – for example, as part of the Basel initiatives on effective risk data aggregation and risk reporting – some important areas still need to be amended or adapted to ensure compliance with requirements on data quality and management. These areas include data quality monitoring and internal control, and the allocation of roles and responsibilities relating to data management.

11 Given the nature of the portfolios under review in terms of limited off-balance sheet exposures, the

analysis of the findings focused on the PD and LGD parameters. 12 These findings cover the PD and LGD parameters as these were the parameters most frequently

assessed during TRIM investigations. 13 The rating assignment process was not assessed in investigations covering credit risk models related to

retail and SME portfolios. This is because for these portfolios the rating assignment is often performed on an automated basis.

14 When the LGD parameter was within the scope of the investigation.


6

• Market risk models: the greatest number of findings related to the value-at-risk (VaR) and stressed value-at-risk (sVaR) methodology, regulatory back-testing and the scope of the internal models approach (IMA)15. In particular, about 60% of the TRIM investigations on market risk resulted in at least one high-severity finding on the VaR and sVaR methodology. Furthermore, just over 80% of SIs that used incremental default and migration risk charge (IRC) models received at least one high-severity finding in relation to those models.

• Counterparty credit risk models: validation and governance were the topics with the highest number of findings (all of the counterparty credit risk investigations featured at least one finding related to these topics and in 60% of cases, the findings were raised with a high severity). There were also findings on specific modelling topics such as trade coverage, the margin period of risk, collateral, initial margin, and risk factors and calibration.

• Topics related to non-model-specific aspects, with a focus on credit risk (“general topics”): most findings related to the organisation and activities of the internal validation function, roll-out and permanent partial use (PPU), and the management of model changes. Notably, all institutions in the scope of the general topics review received feedback letters with recommendations16 that indicated that parts of their practices were not in line with the ECB’s understanding of applicable regulatory requirements. A subset of institutions received a supervisory decision17 containing obligations18 to address deviations from the applicable regulatory requirements.

Further details on the main observations and findings from the on-site investigations can be found in Section 4 of this report.

The TRIM project has led to detailed supervisory follow-up with the involved institutions. Institutions are expected to work intensely to address the findings raised and to ensure that these remedial actions also take appropriate account of the new requirements stemming from the EBA’s regulatory review of the IRB approach.

The supervisory follow-up depends on the nature of the deficiency to be addressed. Institutions can be asked to improve their documentation, conduct additional analyses or to better justify their approaches, where required. If there are specific concerns

15 The IMA is an approach available to institutions under which they may use internal models approved by

banking supervisors in the calculation of their own funds requirements for market risk. 16 A recommendation is an action recommended to the institution. Unlike an obligation it has no legally

binding effect or deadline. See also the Guide to on-site inspections and internal model investigations on the ECB’s banking supervision website.

17 A supervisory decision is a legal act adopted by the ECB in the exercise of the tasks and powers conferred on it by the SSM Regulation and is usually addressed to a credit institution. The decision grants rights and/or imposes obligations modifying the situation of the addressee. The decision may include ancillary provisions such as time limits, conditions, obligations or non-binding recommendations. See also the SSM Supervisory Manual on the ECB’s banking supervision website.

18 An obligation is a remedial action imposed on an institution in order to achieve compliance with a legal requirement, without postponing the application of the permission or limiting the (permitted) use of the internal model. The action the institution is expected to carry out shall be specified in a sufficiently detailed manner. If there are several addressees, it must be clear on which addressee an individual obligation is imposed. Obligations have a deadline by which they must be met, or the ECB may revoke the permission. Obligations must correspond to one or more findings. See also the Guide to on-site inspections and internal model investigations on the ECB’s banking supervision website.

The findings communicated within TRIM have been followed up with binding supervisory decisions requesting the institutions to address these shortcomings within set timelines


https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.supervisorymanual201803.en.pdf?42da4200dd38971a82c2d15b9ebc0e65




7

about a lack of evidence of compliance, institutions can be asked to change or explain that aspect. If any part of an institution’s approach is clearly not in line with regulatory requirements, the institution is required to return to compliance. Finally, if an institution’s approach does not take into account upcoming requirements, a non-binding recommendation is issued.

Section 5 provides an overview of the use of obligations. An interesting pattern was that in areas in which institutions are given the freedom to design their own approaches, the obligations often require banks to better document, analyse and justify aspects of their approaches. However, in areas where the regulations are more prescriptive, for example in the calculation of the default rate, institutions are more likely to be asked to change that specific aspect, which may have a direct impact on risk parameters and, in turn, capital requirements. More than 40% of the obligations imposed have an implementation time frame of between 12 and 18 months after the issuance of the respective decision, while for a quarter of the obligations this time frame is less than 12 months. For the remaining obligations, the time frame is greater than 18 months.

Since it takes time for institutions to correct these deficiencies, whenever it was detected that non-compliance led to an underestimation of capital requirements, immediate action was taken, e.g. through limitations19 or requiring a more conservative calculation.

As a follow-up to the TRIM investigations, 253 supervisory decisions have been issued or are in the process of being issued. Out of this total, 74% contain at least one limitation and 30% contain an approval of a material model change. It is estimated that the aggregated impact of TRIM limitations and model changes approved as part of TRIM investigations will lead to a 12% increase in the aggregated RWA covered by the models assessed in the respective TRIM investigations. This corresponds to an overall absolute increase in RWA of about €275 billion as a consequence of TRIM and to a median impact of -51 basis points and an average impact of -71 basis points on the CET1 ratios of the in-scope institutions.

Going forward, institutions need to continue to work on their internal models to maintain the high quality of models achieved through TRIM. This includes defining internal model strategies for the development and maintenance of internal models. In particular, the independent internal validation function needs to be further strengthened in line with the TRIM requirements so as to ensure an ongoing internal challenge of the performance of internal models and appropriate follow-up of remedial actions. Defining these internal model strategies will also support institutions in the decision on the optimal use of time and resources invested in model development and maintenance, and it may lead to simplifications in the current model landscapes as part of the preparation for upcoming regulatory developments – which foresee, in particular, the decommissioning of some existing models – or to the corresponding necessary improvements also for some less material or less critical models.

19 Each limitation should explicitly describe when it will no longer apply. Limitations are usually linked to one

or more obligations – generally with deadlines that are identical or close to each other – and, in these cases, the limitations apply until the ECB confirms in writing that the linked obligations have been fulfilled.

It is estimated that the aggregated impact of TRIM limitations and model changes approved as part of TRIM will lead to a 12% increase in the aggregated RWA covered by the models in scope of TRIM and an overall absolute increase in RWA of about €275 billion

Going forward, institutions need to continue to invest in the development and maintenance of their internal models


8

In the process of pursuing the primary objectives of reducing non-risk-based RWA variability and harmonising supervisory practices, TRIM developed overarching approaches which have contributed to a consistent outcome of the TRIM exercise and which, going forward will further foster consistent and high-quality supervision of the internal models used by SIs, contributing to a level playing field.

• Through the development of the ECB guide to internal models, TRIM has contributed to a common understanding across the European banking supervision of regulatory requirements relating to internal models. This common understanding is a key prerequisite for ensuring the use of consistent supervisory practices and standards within the SSM and, as such, its importance cannot be overestimated. The guide will remain as a reference document after the finalisation of the TRIM project and may also be updated in the future, as necessary, to reflect new regulatory developments or to cover additional topics that are currently not included.

• A standardised and thorough approach to assessing internal models has been designed and systematically rolled out in the context of TRIM. The use of standardised data requests as well as common inspection techniques and tools (ITTs) by the various on-site inspection teams involved in TRIM has helped to effectively translate and implement the harmonised understanding of regulation into a consistent and transparent approach to IMIs, including the production of comparable assessment reports. The assessment methodology to be deployed in future IMIs will also greatly benefit from the techniques developed within TRIM.

• An extensive on-site approach has been implemented. The allocation of a sufficient number of resources allowed a large number of on-site IMIs to be carried out.

• Finally, TRIM has provided a systematic overview of the key features and weaknesses of the internal models in use at SIs. The horizontal analyses carried out in the context of TRIM, summarised in Section 4 of this report, have enabled the ECB to identify the most common or critical shortcomings of internal models assessed during on-site investigations. This formed the basis for a consistent supervisory follow-up and allowed the identification of areas which may require particular attention in future internal model supervision.

From a supervisory perspective, the depth and breadth of the knowledge generated by TRIM through this systematic overview of key features and shortcomings of the most critical and material models across the European banking supervision is in itself an unprecedented achievement of the project. Going forward, this provides a powerful tool for supervisors, particularly, when deciding on supervisory strategies and priorities.

Moreover, TRIM further demonstrated the appropriateness and effectiveness of the concerted supervisory approach – leveraging on common ITTs and thorough on-site IMIs – for an effective assessment of the adequacy of internal models. Therefore, the supervision of internal models used by SIs will continue to rely on the key pillars of intrusive (on or off-site) IMIs, supplemented by the insights from ongoing model

Under TRIM, overarching approaches have been developed which have contributed to the fulfilment of the objectives of reducing non-risk-based RWA variability and harmonising supervisory practices


9

monitoring. While on-site investigations are a very powerful tool for in-depth analysis of the compliance of internal models with regulatory requirements, rigorous off-site assessments can complement the on-site approach for less material or less critical models, or model changes, by allowing for a more risk-based approach. In addition, ongoing model monitoring tools (such as benchmarking or the review of validation reports) can also help to identify areas where further investigations are needed. Furthermore, the systematic assessment of models for similar portfolios (e.g. in the form of investigation “campaigns”) should be maintained, where appropriate, in order to allow comparison and harmonisation.

Going forward the positive experience and wealth of results from TRIM will bring long-lasting and sustainable benefits beyond the project itself, not least by helping institutions to be better prepared to face current and future challenges related either to economic shocks or to the adaptation of existing models to regulatory developments. In fact, the implementation of Basel III standards, through the amendments to the Capital Requirements Regulation (CRR), the regulatory review of the IRB approach led by the EBA and the remediation of TRIM findings should provide complementary layers of safeguarding against inadequate internal models.

Finally, TRIM substantially contributed to ensure that institutions are compliant with the applicable regulatory requirements on internal models, meaning for instance that the key parameters of credit risk models are estimated by also taking into account downturn conditions. This feature of the risk estimates, accompanied by appropriate rating assignment dynamics, should allow models to perform adequately under different economic conditions and at different points in the business cycle. Thus, good internal models help to measure risk appropriately, thereby better preparing banks to react and answer to economic shocks like the COVID-19 pandemic.

Going forward, the experience and results of TRIM will bring long-lasting and sustainable benefits beyond the project, by helping institutions to be better prepared to face current and future challenges

Targeted Review of Internal Models – Purpose and scope of the report

10

2 Purpose and scope of the report

This report is based on the final outputs of TRIM and contains a summary of the project’s key activities and results. The intention is to provide the reader with an understanding of what TRIM has delivered and how this has been used to support the project’s objectives of assessing the internal models used by SIs within the SSM and of developing and applying harmonised supervisory practices.

The content of this report is divided into four main subjects (each with its own section):

• an overview of the TRIM project, including its objectives, scope, structure and operational activities (Section 3);

• a summary of findings and key observations per risk type (Section 4);

• an overview of the supervisory follow-up (Section 5);

• a project review of TRIM, including achievements, lessons learned and recommendations (Section 6).

Targeted Review of Internal Models – Overview of the TRIM project

11

3 Overview of the TRIM project

3.1 TRIM context and objectives

Under the standards20 issued by the BCBS, as implemented in European Union (EU) legislation21, banks are allowed to employ internally developed models (“internal models”) for the purpose of calculating regulatory capital requirements, provided these have received supervisory approval. Following the financial crisis of 2007-2009, concerns were raised regarding the unwarranted (i.e. non-risk-based) variability of outputs of some models across banks, alongside criticism from external stakeholders of the complexity of the models and the resulting opaqueness of the modelling approaches.

These concerns led to further regulatory work by the BCBS, supplemented in the EU by initiatives by the EBA, such as the regulatory review of the IRB approach to credit risk22, which sought to reduce non-risk-based variability in model outcomes through regulatory guidance and clarifications in order to ensure the comparability of risk estimates while at the same time preserving risk sensitivity. Also through its annual assessment of the consistency of internal model outcomes, the EBA assesses how to explain risk-weights variability for credit and market risks. In December 2017 the BCBS published its finalisation of the Basel III reforms23, which included restrictions24 across different risk types on the use of internal models in the areas considered to contribute significantly to excessive variability of risk exposure amounts (or RWA).

In conjunction with these regulatory initiatives, the ECB’s direct supervision of SIs under the SSM has provided a unique opportunity to improve the consistency of internal models across the euro area. The TRIM project was a large-scale multi-year supervisory initiative launched by the ECB at the beginning of 2016 in close cooperation with NCAs in order to:

• confirm the adequacy and appropriateness of approved Pillar I internal models used by SIs in euro area countries, ensuring their compliance with regulatory requirements;

• harmonise supervisory practices relating to internal models within the SSM.

20 For more information, see “The Basel Framework” on the Bank for International Settlements’ website. 21 See Regulation (EU) No 575/2013 of the European Parliament and of the Council of 26 June 2013 on

prudential requirements for credit institutions and investment firms and amending Regulation (EU) No 648/2012 (OJ L 176, 27.6.2013, p. 1) (Capital Requirements Regulation – CRR) and Regulation (EU) 2019/876 of the European Parliament and of the Council of 20 May 2019 amending Regulation (EU) No 575/2013 as regards the leverage ratio, the net stable funding ratio, requirements for own funds and eligible liabilities, counterparty credit risk, market risk, exposures to central counterparties, exposures to collective investment undertakings, large exposures, reporting and disclosure requirements, and Regulation (EU) No 648/2012 (OJ L 150, 7.6.2019, p. 1).

22 For general principles and timelines concerning the implementation of the regulatory review of the IRB approach, see the EBA Opinion on IRB implementation (EBA/Op/2016/01), published in February 2016, which is available on the EBA’s website. A progress report on the IRB roadmap, published in July 2019, is also available on the EBA’s website.

23 See Basel III: Finalising post-crisis reforms, BCBS, December 2017. 24 Not yet implemented at the time of publication of this report.

The use of internal models to calculate capital requirements led to concerns around unwarranted (non-risk-based) variability of their outcomes

Regulators have commenced various initiatives to address such concerns

The ECB has launched the TRIM project to confirm the adequacy of internal models of SIs and to further harmonise supervisory practices

https://www.bis.org/basel_framework/










https://eba.europa.eu/eba-sets-out-roadmap-for-the-implementation-of-the-regulatory-review-of-internal-models

https://eba.europa.eu/eba-publishes-report-on-progress-made-on-its-roadmap-to-repair-irb-models

https://www.bis.org/bcbs/publ/d424.pdf


12

Unwarranted or non-risk-based RWA variability can arise from various sources, e.g. from regulation being insufficient to rule out such variability, or from institutions not complying with regulatory requirements. This report focuses on differences in the implementation of existing regulation as a driver of unwarranted RWA variability. These differences can stem from non-compliance with regulatory requirements or, in some cases, developments in supervisory practices over time.

Within this mandate, TRIM contributed to reducing unwarranted variability of RWA and to maintaining a level playing field among SIs for the use of internal models, thereby also increasing the credibility of their outcomes.

TRIM should be seen in the context of ECB Banking Supervision’s overall approach to the supervision of internal models. The main components of this approach are:25

• IMIs, which may be initiated by the ECB (as in the case of TRIM) or triggered by a request from an institution for supervisory approval of a new model, a material model change, or an extension of an existing model;

• ongoing model monitoring, which is used to verify on an ongoing basis the performance of institutions’ models and their compliance with regulatory requirements.

It should be emphasised that, while TRIM was temporary in nature, it is expected to have a lasting and positive impact on both components. By harmonising supervisory practices, the comparability and consistency of IMIs across countries is enhanced. Furthermore, by clarifying expectations regarding the continuous maintenance of internal models by institutions, the ongoing model monitoring pillar is also strengthened.

With activities stretching over more than four years, TRIM is the largest project so far conducted by ECB Banking Supervision in coordination with NCAs. It marks an important milestone in raising the quality standards and comparability of outcomes of internal models in use at SIs within the SSM. At the core of TRIM was the execution of 200 on-site IMIs across 65 institutions. The decision to rely primarily on on-site investigations for TRIM stemmed from the consideration that on-site investigations are the most effective tool to confirm the adequacy of internal models, as they provide a thorough and intrusive procedure for model assessment. Owing to the large number of approved internal models at SIs and the time needed for on-site investigations, TRIM adopted a targeted approach so as to review those topics deemed to contribute significantly to unwarranted RWA variability and, for credit risk, defined a scope of review based on the models deemed most critical and material.26 The considerations used to define the scope of the TRIM project are outlined further in Section 3.2 of this report.

25 For more information, see the “Internal models” page on the ECB’s banking supervision website. 26 In this context, criticality reflects the performance of the model, whereas materiality is based on the

underlying exposure at default covered by the model. See Section 3.2.3 for further details.

TRIM is a key initiative that has contributed to maintaining a level playing field among SIs and has had a lasting and positive impact on the approach to supervising internal models

TRIM is the largest project conducted by ECB Banking Supervision so far

https://www.bankingsupervision.europa.eu/banking/tasks/internal_models/html/index.en.html


13

To ensure consistency and to support the execution of on-site IMIs, TRIM has delivered the ECB guide to internal models (the “ECB guide”)27, which contributes to a level playing field for SIs in the area of internal models by harmonising supervisory practices and providing transparency on the ECB’s supervisory understanding of existing regulation concerning the topics under review in the TRIM project. In addition, a common methodological approach – ITTs28 – was developed and used in on-site investigations to ensure a consistent assessment approach.

Furthermore, internal ECB Banking Supervision mechanisms (e.g. several layers of managerial quality assurance, horizontal analyses and approval of supervisory decisions in batches to allow comparisons across institutions) were used to ensure consistent results of TRIM on-site investigations across institutions, as well as a consistent follow-up via supervisory decisions or, where appropriate, operational acts.

Finally, the TRIM project has created organisational structures and processes which ensure harmonisation by means of close cooperation among all the parties involved, gathering skilled internal model experts from across the European banking supervision. Where applicable, these structures and processes will be implemented in future supervision of internal models, thereby strengthening the supervision and providing a long-lasting benefit.

Further details of the TRIM project activities and deliverables are provided in Sections 3.3 and 3.5 of this report.

3.2 Scope of TRIM

All SIs with approved internal models fell within the scope of the TRIM project, with limited exceptions to reflect structural changes, such as mergers and acquisitions. However, considering the large number of internal models used by SIs (in particular for credit risk), rather than a full-blown review of all existing and approved models, TRIM focused on a targeted approach to internal model assessment. This approach was based on the following principal characteristics:

• covering the most relevant risk types, taking further regulatory developments into account;

• covering the most significant areas in which unwarranted RWA variability was deemed to exist;

• for credit risk, reviewing a sufficiently large number of the most material and critical internal models, subject to a proportionate use of time and resources.29

Within each risk type investigated, the areas that were selected for investigation are also reflected in the ECB guide. Areas such as the own funds requirements calculation 27 See ECB guide to internal models, ECB, October 2019. 28 A set of common methodologies and checks developed for deployment in TRIM investigations in order to

ensure a consistent assessment approach and supervisory stance across teams. 29 For market risk and counterparty credit risk, all models were covered, with the exception of correlation

trading models for market risk and credit valuation adjustment models for counterparty credit risk.

A key deliverable of the TRIM project was the ECB guide to internal models, which contributes to the harmonisation of supervisory practices and provides transparency to institutions

Several layers of quality assurance were used to ensure consistent results and supervisory follow-up of TRIM investigations

TRIM provided for a system-wide in-depth assessment of internal models, relying on a targeted approach to maximise its effectiveness

https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.guidetointernalmodels_consolidated_201910%7E97fd49fb08.en.pdf


14

or the assignment of exposures to exposure classes, while relating to internal models, were outside the scope of the TRIM on-site investigations.

Overall, TRIM has provided a very granular system-wide review of the internal models used by SIs. The targeted approach deployed in TRIM ensured the feasibility of accomplishing such a complex project in a relatively limited period of time. It could be adopted in view of the strong integration of TRIM within the regular model supervision performed by the ECB, which ensures that over time SIs or specific models that were not assessed in TRIM will be reviewed according to the same principles.

3.2.1 In-scope institutions

In principle, all SIs which at the start of the project had approved internal models for the in-scope risk types were within the scope of TRIM. However, a small number of adjustments were made to this set of institutions. For example, institutions undergoing a merger or which would no longer be subject to direct supervision by the ECB were not included, while institutions that did not have an approved internal model at the start of the TRIM project, but which received approval during the execution of the project, were included for assessment where possible. This resulted in a total of 65 SIs being within the scope of the TRIM project.

A breakdown by country of establishment of the institutions assessed in the TRIM project is provided in Figure 1.

Figure 1 Institutions within the scope of TRIM

Code Country SIs in scope

AT Austria 3

BE Belgium 5

DE Germany 14

ES Spain 6

FR France 8

GR Greece 3

IE Ireland 4

IT Italy 7

NL Netherlands 4

Other Other 11

Total Total 65

Source: ECB. Note: “Other” includes euro area countries with no more than two SIs within the scope of TRIM.

The 65 institutions within the scope of TRIM account for more than 85% of the total assets of all SIs within the SSM (as at the end of 2019).

5

4

14

3

7

6

8

4

3

+11 Other

In total, 65 significant institutions were included within the scope of TRIM, i.e. all SIs with internal models, with limited exceptions


15

3.2.2 Risk types

In theory, all Pillar 1 approved models could contribute to non-risk-based RWA variability. However, for the purposes of TRIM, the scope was confined to the most material models, while also reflecting the environment of ongoing regulatory developments at the time of the project (in particular in relation to operational risk).

The scope of TRIM was based on internal models for the following Pillar 1 risks:

• credit risk;

• market risk;

• counterparty credit risk (CCR).

Finally, operational risk, while being a Pillar 1 risk type, was excluded from TRIM in view of the BCBS’s stance that, as part of the reforms for finalising Basel III, banks should not use internal models for this risk type in the future.30 As this reform is expected to be transposed into Union law under a revised CRR, it was decided to exclude operational risk models from the scope of TRIM.

Similarly, models relating to credit valuation adjustment (CVA) risk were also outside the scope of the TRIM project, again taking into account the BCBS’s stance against the use of internal models for this risk type.

3.2.3 Selection of models for on-site investigation

As previously noted, TRIM covered three risk types: credit risk, market risk and CCR. In the case of market risk and CCR, all approved internal models31 in place at the inception of TRIM (in 31 and eight SIs, respectively) were able to be reviewed within the project. However, the number of approved credit risk models was too large to allow complete coverage within TRIM, so a more targeted approach towards models for this risk-type was adopted. A dedicated workstream (“centre of competence”)32 on model map and prioritisation was tasked with developing the approach to the prioritisation and selection of credit risk models for on-site investigation.

The selection of the credit risk models for on-site investigations was based on the following main criteria:

• materiality, in particular by reference to the exposure at default (EAD) covered by the model;

30 An example of this stance can be found on page 8 of “High-level summary of Basel III reforms”, BCBS,

December 2017. 31 With the exception of correlation trading and CVA models. 32 Each centre of competence was a functional workstream for the delivery of TRIM project content. TRIM

included six centres of competence for: (i) general topics; (ii) credit risk models for LDPs; (iii) credit risk models for retail and small and medium-sized corporate portfolios; (iv) market risk; (v) CCR; and (vi) model map and prioritisation. See also Section 3.3.1.

TRIM targeted the most material models, except in areas where the use of models was expected to be discontinued

Consistent criteria were applied to select the credit risk models to be investigated in TRIM

Materiality and criticality were key criteria for selecting the credit risk models

https://www.bis.org/bcbs/publ/d424_hlsummary.pdf


16

• criticality, referring to the perceived performance of the model (as determined by, for example, the institution’s internal validation or internal audit activities, the EBA benchmarking exercise, supervisory judgement and selected quantitative measures).

Several sources of information were used to determine the most material and critical models, including a supervisory survey, a dedicated models questionnaire sent to institutions and results from previous EBA benchmarking exercises. These were complemented by considerations regarding whether the selected models were of particular supervisory interest, as determined by findings from previous IMIs and feedback from the relevant Joint Supervisory Team (JST) or NCA. Expected model changes were also taken into account, allowing the merging of some TRIM investigations with the supervisory assessment of material model changes.

This selection process for credit risk models was designed so that the sample of models investigated under TRIM would be sufficiently representative of the overall population of approved internal models, and so that the sample could support meaningful horizontal analyses. This resulted in an overall coverage of credit risk portfolios in the TRIM investigations corresponding to 66% of the aggregate EAD of SIs related to portfolios within the scope of TRIM (i.e. excluding exposures for which capital requirements are calculated using the standardised approach, or types of exposure that were excluded from TRIM owing to expected regulatory developments, such as central government or central bank exposures, equity exposures, and exposures covered by the slotting approach for specialised lending).33

Owing to the large number of credit risk models to be reviewed, the investigations were divided into two parts: one part dealing primarily with models related to retail and small and medium-sized corporate portfolios (“retail and SME” portfolios)34, and another part mainly focusing on models for exposures to medium-sized and large corporates, institutions35, and specialised lending (“low-default” portfolios or LDPs).36

Notwithstanding this organisational division, the focus of TRIM (both in the investigations and in the corresponding supervisory decisions) was in all cases on the internal models used by the institutions, rather than on specific portfolios or exposure classes.

Altogether, a total of 200 on-site investigations were performed within TRIM. Table 1 provides a breakdown of the investigations by risk/exposure type.

33 This coverage figure is based on the EAD disclosed in the assessment reports for the PD and LGD

models investigated in TRIM relative to the total EAD of in-scope portfolios as at the most recent credit risk investigation for each institution.

34 In this report, “SME” (small and medium-sized enterprise) is used in a broader sense than defined in regulatory texts to match more closely the way the term is used by institutions within the scope of TRIM.

35 “Exposures to institutions” are those exposures defined under Article 147(4) of the CRR. 36 This organisational division was only for simplicity, and the final allocation of models to be reviewed in

each of the two parts of the project could deviate from this general criterion as needed. In particular, there was no formal definition of “low-default portfolio”. Some models assigned to the LDP category were not strictly LDP models, while others assigned to the “retail and SME” category had the characteristics of LDP models.

Overall coverage of credit risk portfolios in TRIM investigations corresponded to 66% of the aggregate EAD of SIs related to portfolios within the scope of TRIM

TRIM entailed the execution of 200 model investigations


17

Table 1 Number of TRIM on-site investigations

Risk/exposure type Number of investigations Percentage of total

Credit risk – retail and SME 85 42.5%

Credit risk – LDP 76 38.0%

Market risk 31 15.5%

Counterparty credit risk 8 4.0%

Total 200 100%

Source: ECB.

Where possible, to realise synergies and increase efficiency, TRIM investigations were merged with regular IMIs to assess institutions’ requests for supervisory approval of model changes, at the initiative of the institution (“TRIMIX” investigations).37

In addition to the risk/exposure types noted above, a separate TRIM subject area (“general topics”) was established in order to evaluate non-model-specific matters pertaining to the “environment” for the use of internal models (e.g. internal model governance, internal validation and internal audit), with a focus in particular on topics related to credit risk, given the materiality of this risk type in terms of own funds requirements (see Section 4.1).

3.3 Project roadmap and communication

The TRIM project was divided into three phases: a preparation phase, an execution phase, and a finalisation phase. Figure 2 shows the main stages and activities of these phases, which are described in more detail in the following sections.

37 In total, 81 out of 200 TRIM on-site investigations were conducted as TRIMIXs. These comprised

36 investigations of credit risk models for retail and SME portfolios, 24 of credit risk models for LDPs, 13 of market risk models and eight of counterparty credit risk models.

A dedicated “general topics” review was performed to evaluate non-model-specific areas such as internal validation and governance, with a focus on credit risk


18

Figure 2 TRIM project roadmap

Source: ECB. Notes: The operational relief granted to institutions in March 2020 by ECB Banking Supervision in response to the COVID-19 outbreak included a six-month postponement of the issuance of TRIM decisions, on-site follow-up letters and internal model decisions not communicated to institutions by that date. The ECB resumed supervisory processes for adopting new decisions on TRIM, on-site follow up letters and internal model decisions from October 2020. For further details on the supervisory decision process, see Sections 3.4.1 and 3.5.

3.3.1 Preparation phase

The TRIM preparation phase was launched in January 2016 and lasted until the first quarter of 2017. After defining the scope of Pillar 1 risks to be considered and the broad governance structure for TRIM, the preparation phase focused on the following:

• establishing the steering body for the project (the Harmonisation Board), which was composed of senior model experts and management from the ECB and NCAs and was responsible for the oversight and vetting of all deliverables of the TRIM project and related supervisory stances (see Section 3.4);

• establishing the centres of competence for credit risk, market risk, CCR, general topics, and model map and prioritisation, which comprised groups of internal model experts from both the ECB and NCAs and were responsible, inter alia, for developing the ECB guide, defining the methodology used in TRIM, providing methodological support to the assessment teams, and conducting horizontal analyses of the outcomes of on-site investigations (see Section 3.4);

• setting up horizontal processes, such as quality assurance, monitoring and reporting, to support the execution of on-site investigations;

In-scope models for investigation

determined

Public consultation of the ECB guide

carried out

TRIM on-site logistics and processes established

Inspection techniques and tools developed

General topics horizontal analysis

performed

Q2 2017Launch of on-site

investigations

Q4 2019Completion of

TRIM on-site work

Project report published

Ongoing monitoring and administration of investigations

performed

On-site investigations and general

topics review

Project framework and

organisation formalised

Preparation Execution

Q1 2016TRIM project

initiation

Q2 2021Closure of

TRIM project

Finalisation

First version of the ECB guide

developed

Consistency checks in relation to the on-site investigations carried out

On-site investigations carried out and assessment reports produced

Horizontal analysis of findings and observations

performed

Supervisory decisions prepared for adoption by the Governing Council and

institution follow-up

ECB guide to internal models

Project management

activities

Consolidated version of the

guide published

Supervisory decisions

The TRIM preparation phase (Q1 2016 to Q1 2017) was used to establish appropriate governance structures and processes to support the model investigations, as well as to define a common methodology and understanding to ensure a con-sistent assessment approach across institutions


19

• defining the methodological topics to be assessed (based on the expected main drivers of unwarranted RWA variability);

• developing a first version of the ECB guide (initially published as “Guide for the TRIM”), setting out the ECB’s understanding of the regulatory framework for internal models, in particular relating to those topics deemed to contribute mostly to non-risk-based variability in model outcomes;

• defining ITTs – a common methodology, standardised data requests and supporting tools to be applied consistently by the various assessment teams conducting TRIM on-site investigations;

• carrying out a large data collection exercise, and analysing the information gathered to refine the view of RWA variability drivers and to determine the most material and critical credit risk models to target in the on-site investigations;

• evaluating institutions’ practices under general topics.

3.3.2 Execution phase

Following the preparatory work, the execution phase of TRIM started in the second quarter of 2017 with the aim of conducting 200 on-site investigations across the in-scope institutions. The on-site investigations were carried out on the basis of the common methodology defined in the preparation phase.

For simplicity, the TRIM on-site investigations can be divided into two parts:

• part one (from the second quarter of 2017 to the second quarter of 2018) mainly involved a review of the internal credit risk models for retail and SME portfolios, as well as market risk and CCR models;

• part two (from the third quarter of 2018 to the fourth quarter of 2019) focused primarily on models used to assess credit risk for LDPs.

During the execution phase, different layers of horizontal processes for quality assurance and on-site management (such as monitoring and reporting) were put in place to provide the necessary support to the assessment teams carrying out the investigations and ensure consistent outcomes of their reviews. The different levels of quality assurance that were used played an important role in ensuring a level playing field for the IMIs (see Section 3.4).38

In addition to these on-site investigations, the centre of competence for general topics finalised its horizontal analysis of information collected for its sub-topics, in particular the documentation and responses received from a questionnaire sent to institutions during the preparation phase, as well as information gathered through short supervisory visits to each in-scope institution.

38 These quality assurance processes are also described in “TRIM: reviewing internal models”, ECB,

November 2018.

During the execution phase (Q2 2017 to Q4 2019), 200 on-site investigations were performed to assess internal models

A strong quality assurance framework with complementary layers of horizontal processes was implemented to ensure consistent outcomes of TRIM investigations

https://www.bankingsupervision.europa.eu/press/publications/newsletter/2018/html/ssm.nl181114_4.en.html


20

As part of the normal supervisory follow-up of IMIs, after the completion of each TRIM on-site investigation, institutions were informed of the findings and, through supervisory decisions, were requested to remediate any identified shortcomings (further details on this process are provided in Section 3.5).

In parallel, two public consultations were conducted on different chapters of the ECB guide and a revised version of the guide was published, taking into account the outcome of these consultations (see Section 3.3.4).

3.3.3 Finalisation phase

Following the on-site investigation activity and completion of the consistency checks on the corresponding assessment reports, further activities were performed in order to finalise the TRIM project. These included the finalisation of all horizontal analyses, further issuance of supervisory decisions, and the closure of all internal documentation and reports.

3.3.4 TRIM communications

Throughout each of the project phases, providing transparency on the purpose and progress of TRIM was a key priority for the project. As such, communication activities were an important part of the project management of TRIM and were designed to maintain a continuous dialogue with the institutions involved in the project and provide relevant information to the wider public. This objective was achieved through a variety of means.

First, it was achieved through key project deliverables, such as the ECB guide to internal models, which sets out the ECB understanding of applicable requirements for internal models. The development of the ECB guide was enhanced through the public consultations held on different chapters of the guide, allowing feedback and insights from different institutions and banking associations to be collected, and enhancing transparency.

Second, there was ongoing dialogue with and a flow of information to the institutions within the scope of the project. The exchange was initiated through presentations on the TRIM project at industry conferences organised by the ECB in February 2016 and February 2017, and it continued with regular status updates provided in the form of information letters to institutions and publications on the ECB’s banking supervision website. The letters covered a range of different topics, including general updates on the progress of TRIM and outlooks on next steps, overviews of the upcoming TRIM investigations to be conducted at each institution, and practical information on the logistics and requirements for the preparation of those investigations. As the execution phase progressed, some of the information letters also provided interim updates on the outcomes of TRIM investigations. These letters were also published on the ECB’s

Institutions were informed of the findings of the model investigations and the necessary remediation activities through the well-established supervisory follow-up process also applied to normal IMIs

Communication activities through various channels were a priority of TRIM to ensure full transparency on the progress and outcomes of the project

The ECB guide was a key TRIM deliverable, providing transparency on the ECB’s understanding of the applicable regulatory requirements

An ongoing dialogue with and provision of information to institutions over the course of TRIM facilitated a smooth execution and follow-up


21

banking supervision website to ensure full transparency for the general public regarding these interim results.39

Third, this report completes the dedicated communication process for TRIM. Interaction with institutions regarding the follow-up of TRIM outcomes will continue as part of the regular supervisory dialogue.

3.4 TRIM project governance

The TRIM governance set-up was defined at the beginning of the project in accordance with the following key principles:

1. a streamlined governance structure with clearly defined roles, responsibilities and interactions between structures, including the ECB’s Internal Models Division40 as project lead, the Harmonisation Board as the steering body for the project, and the Supervisory Board as the body approving the main project deliverables and, in particular, final drafts of supervisory decisions (which are then adopted or rejected by the Governing Council);

2. involvement of different NCAs working closely with the ECB at all levels in order to avoid possible national bias and to leverage the wide range of internal model expertise across the SSM;

3. alignment and, where appropriate, integration with general ECB Banking Supervision processes and decision-making (e.g. alignment with the existing ECB framework for IMIs and related decisions).

The TRIM governance structure was designed to leverage subject matter expertise and best practices from across the SSM for content-related matters and provide an appropriate degree of central steering and oversight for project decisions. Examples of how this was implemented in the TRIM project include the following:

• The TRIM centres of competence were the methodological backbone of the project and were responsible, inter alia, for leading the development of the ECB guide and the common methodology applied by the on-site assessment teams. Each centre of competence included experienced internal model experts from the ECB and NCAs.

• The Harmonisation Board, a dedicated operational steering body composed of senior model experts and management from the ECB and NCAs, was responsible for the oversight and vetting of the main deliverables of the TRIM project and related supervisory stances prior to their Supervisory Board approval,

39 See, for example, the update letter of 21 November 2019. 40 Towards the end of the TRIM project the ECB banking supervision underwent a reorganisation which

went live on 1st October 2020 (see announcement). However, reference in this document is still made to the previous organisation structure since this was the one in place for almost all of the life cycle of TRIM.

The governance structure for TRIM featured close cooperation with and involvement of NCAs…

… leveraging subject matter expertise from across the SSM, under the project leadership of the ECB’s Internal Models Division

https://www.bankingsupervision.europa.eu/press/letterstobanks/shared/pdf/2019/ssm.TRIM_information_leteter_201911.en.pdf?9a9b7175dd628153a8774b009c5f3885

https://www.bankingsupervision.europa.eu/press/pr/date/2020/html/ssm.pr200729%7Ee5c783c499.en.html


22

with the exception of individual decisions.41 It served in particular to exchange and balance views from all involved SSM stakeholders.

• A centralised project management team was responsible for monitoring and management activities within the TRIM project, in particular with respect to project planning, the execution and coordination of the on-site investigations, resource management and communication.

The ECB’s Internal Models Division was responsible, inter alia, for chairing the Harmonisation Board, co-chairing and contributing to the staffing of the centres of competence, staffing and leading the project management office (PMO), checking the consistency of assessment reports jointly with the centres of competence, and preparing drafts of all of the supervisory decisions issued as a follow-up to TRIM findings.

Furthermore, a number of additional ECB functions supported the execution of the TRIM project and its compliance with ECB internal procedures, facilitating and providing expert support on topics such as planning and coordination, procurement, legal matters and communication.

Finally, from a process perspective, a key aspect of TRIM governance was the implementation of a comprehensive set of quality assurance processes and safeguards with the aim of ensuring consistent project outcomes and a level playing field for the institutions within scope of the project. These processes are described in more detail in the following section.

3.4.1 TRIM quality assurance processes

Given the magnitude and complexity of the project, a comprehensive and multi-layered quality assurance framework was put in place to ensure consistent and comparable outcomes across institutions. In particular, there were three main areas in which quality assurance processes were applied in order to ensure that the objectives of the TRIM project could be achieved.

A first layer of quality assurance in the TRIM project was to establish a common, standardised methodological framework to serve as the basis for the harmonised execution of TRIM on-site investigations. This included the following elements:

• the ECB guide to internal models, to foster transparency and common understanding among all parties regarding how the ECB understands the regulatory requirements for internal models;

• common ITTs, required in all on-site TRIM investigations to ensure a uniform approach to and coverage of predefined areas of investigation;

41 The preparation of individual supervisory decisions falls under the remit of the ECB’s Internal Models

Division and the relevant Joint Supervisory Teams.

A comprehensive and multi-layered quality assurance framework was put in place within TRIM

1. Each TRIM investigation was based on a common understanding of the regulatory requirements and applied pre-defined, standardised ITTs


23

• close interaction between the assessment teams and the central risk-specific teams during the execution of each on-site investigation, to ensure that the ITTs were applied consistently.

In addition, a question and answer process was used to support institutions and assessment teams throughout the on-site investigations.

A second layer of quality assurance was implemented on an ongoing basis after each on-site investigation was completed. The assessment report produced for the investigation was checked for consistency by internal model experts from different NCAs and the ECB (via the relevant centres of competence and the ECB’s Internal Models Division) who had a horizontal view of the TRIM investigations for each risk type. This ensured that similar shortcomings gave rise to similar findings.

As an additional method to ensure consistency, any disagreements from the consistency check step were further discussed and resolved within the relevant TRIM governance structure.

A third layer of quality assurance was applied ex post by the centres of competence, which performed cross checks and horizontal analyses across finalised assessment reports and raised additional findings when needed. Under this third layer of quality assurance, the TRIM governance bodies were again involved in the event of any disagreement on findings.

Finally, supervisory decisions were prepared on the basis of the findings of each on-site investigation and the results of the consistency checks and horizontal analyses performed by the centres of competence42 by the ECB’s Internal Models Division. The preparation of the decisions was supported by regular exchange and alignment sessions for the model experts, who also benefited from access to a comprehensive overview of past cases. In addition, there were regular exchanges among experts in the Internal Models Division and specific guidance was given in order to foster the harmonisation of the drafting of the findings and of the related obligations or limitations. Multiple layers of managerial review and challenge were applied within the Internal Models Division, and feedback from the relevant JSTs (which include ECB and NCA staff) was sought before the decisions were sent to institutions. Furthermore, supervisory decisions were normally prepared and approved by the Supervisory Board for adoption by the Governing Council in batches, which allowed a horizontal approach to the decisions and to the treatment of similar findings across supervisory decisions.

3.5 Project deliverables

Figure 3 outlines the process used to create the main deliverables of the TRIM project, such as the ECB guide and assessment reports. These deliverables are described in further detail in the rest of this section.

42 Where it was not possible to include the results of the horizontal analyses in the first supervisory decision,

a second decision was issued.

2. Each assessment report was checked for consistency with the common methodology by internal model experts

3. Cross checks and horizontal analysis were a key part of the quality assurance process

Several control mechanisms were also in place with regard to the preparation of supervisory decisions to ensure consistent treatment of comparable risks and deficiencies


24

Figure 3 Overview of TRIM deliverables

Source: ECB. Note: Areas with significant steering involvement from the governance bodies of the project are indicated with dual-coloured arrows.

Further descriptions of the deliverables shown, as well as the key steps in producing them, are provided in Table 2.

Decision letter

Assessment report

Horizontal analysis

Project report

Horizontal analysis

General topics

Horizontal analysis was performed using the information in the assessment reports

General topics were evaluated on the basis of a comprehensive standardised request for information and documentation, and a short on-site visit (instead of an on-site model investigation)

A summary of observations and findings is included in Section 4 of this project report.

Each on-site investigation was carried out by an assessment team composed of:• ECB staff• NCA staff• External support

Assessment report sent to institution

Credit risk, market risk and counterparty credit risk

Investigations were executed using the pre-defined inspection techniques and tools developed by the centre of competence for that risk type.

Assessment reports

From other investigations

Decision letter

Additional findings identified by horizontal analysis and sent to the institution (if not included in the first letter)

On-site investigations

General topics centre of competence

Supervisory decisions sent to institution

General topics review

Credit risk, market risk, counterparty credit risk and general topics


Centres of competence were responsible for defining the

methodological framework for TRIM

First version of ECB guide

The ECB guide was further updated on the basis of: • feedback from the industry (including public

consultations held in 2018 and finalised in 2019)• experience gained from the on-site investigations and

horizontal analysis



25

Table 2 Main deliverables of the TRIM project

Deliverable Description


The ECB guide to internal models (“ECB guide”) documents how the ECB understands the regulatory requirements for internal models (with a focus on the areas selected for review within TRIM) and provides transparency on how it applies them when assessing whether institutions meet these requirements.43

The guide is composed of four chapters: a general topics chapter, covering non-model-specific requirements (with a focus on credit risk), and three chapters relating to the internal models for credit risk, market risk and CCR.

The guide was developed in alignment with ongoing regulatory developments, such as EBA regulatory technical standards (RTS) and guidelines.

The preparation of the guide benefited from the experience gained from the execution of the TRIM investigations, as well as from public feedback obtained in several waves. The first version of the guide was published on 28 February 2017 for industry comment ahead of the start of on-site investigations. A revised version of the guide was subsequently sent for formal public consultation in 2018.44 Following this consultation process, a revised version of the ECB guide was published in November 2018 (general topics chapter) and July 2019 (risk-type-specific chapters), followed by a consolidated version in October 2019.

The ECB guide will remain as a reference document after the finalisation of the TRIM project. As such, the ECB may update the Guide to reflect possible changes to regulatory requirements and, if applicable, the latest experience gained from IMIs, or to include other topics related to internal models that were not within the scope of the TRIM project.

Inspection techniques and tools (ITTs)

In order to facilitate a consistent and comparable execution of TRIM on-site investigations, standardised data requests and ITTs were drawn up during the preparatory phase of the TRIM project. The ITTs provided assessment teams with a set of relevant inspection methodologies and checks for use in conducting the model investigation (i.e. in order to harmonise and strengthen the assessment approach).

On-site model investigations and assessment reports

On-site IMIs were at the core of the TRIM project, applying a common methodological approach to assess the regulatory compliance of internal models used by SIs. The existing ECB Banking Supervision framework for IMIs45 was used as the basis for the TRIM investigation approach, enhanced with additional processes (for example, relating to dedicated data requests) and the ITTs that were developed for TRIM. Furthermore, existing guidance on the assignment of finding severity and granularity for use in model investigations was further refined in order to harmonise and strengthen the assessment approach across teams.

For each investigation, an assessment report was produced by the assessment team. The assessment report described the activities and results arising from the on-site investigation. After the completion of the on-site phase, each TRIM draft assessment report underwent consistency checks performed (in general) by ECB staff and an NCA other than the NCA involved in the on-site work to ensure a harmonised approach across investigations and a consistent application of the methodology and techniques. Upon completion, the report was provided to the assessed institution and formed the basis of supervisory decisions.

Horizontal analysis Each centre of competence performed a horizontal analysis based on the assessment reports with the aim of forming a comprehensive view of the main model design features (“the modelling landscape”) as well as typical shortcomings of existing models. As such, this analysis also contributed to ex post checks to ensure consistency in the identification and treatment of findings across investigations. In some cases, these ex post checks resulted in additional findings that were reflected in the supervisory decisions.

Horizontal analyses were produced for the different risk-types (credit risk, market risk and CCR) as well as for general topics.

A summary of key information related to the modelling landscape and observations from the horizontal analyses is presented in Section 4 of this report.

43 The guide is not intended to go beyond existing applicable Union and national law or to replace, overrule

or modify applicable Union and national law. 44 Further information, in the form of the ECB’s feedback statement and comments received, can be found

on the ECB’s banking supervision website under “Public consultation on the draft ECB guide to internal models – General topics chapter” and “Public consultation on the ECB guide to internal models – risk-type-specific chapters”.

45 See the Guide to on-site inspections and internal model investigations, ECB, September 2018.

The ECB guide sets out the ECB’s understanding of regulatory requirements for internal models and will continue to serve as a reference document after the finalisation of TRIM

Common inspection methodologies were applied across TRIM on-site investigations The execution of 200 on-site investigations was at the core of the TRIM project to ensure an in-depth review of the models in scope

Horizontal analyses were performed across the assessment reports to ensure consistency of outcomes and provide a comprehensive view of the modelling landscape and any identified shortcomings

https://www.bankingsupervision.europa.eu/legalframework/publiccons/html/internal_models.en.html

https://www.bankingsupervision.europa.eu/legalframework/publiccons/html/internal_models.en.html

https://www.bankingsupervision.europa.eu/legalframework/publiccons/html/internal_models_risk_type_chapters.en.html

https://www.bankingsupervision.europa.eu/legalframework/publiccons/html/internal_models_risk_type_chapters.en.html



26

Deliverable Description

Supervisory decisions

Following the completion of an on-site investigation (for credit risk, market risk or CCR), a process was initiated to provide the institution with supervisory decisions listing the findings and the required remedial action. The process was as follows:

A first supervisory decision with obligations and corresponding deadlines was addressed to institutions after each investigation, requiring the remediation of findings as per the deadlines communicated in the decision. Where severe findings and/or a material underestimation of RWA were identified, limitations46 were imposed in a consistent manner taking into account the specificities of each case. In addition, where applicable, the ECB provided recommendations to institutions with a view to assisting their future compliance with upcoming legal requirements and providing further clarification or specification of existing requirements, e.g. in the light of existing best practices. In order to facilitate an additional layer of horizontal comparison, the draft decisions were normally prepared and approved in batches.

Where necessary, after due consideration of the specific circumstances of each institution and having formed a full horizontal view, a second supervisory decision was sent at a later stage, taking into account (i) the results of horizontal analyses of the outcome of the TRIM investigations, (ii) the finalisation of ongoing regulatory developments, and (iii) the updated ECB guide to internal models.

A second supervisory decision was an instrument envisaged for the TRIM project and was only issued in cases where the relevant supervisory measures could not be included in the first and would typically target only selected topics where a full horizontal view was deemed necessary before initiating the supervisory follow-up with individual institutions (e.g. grade assignment dynamics for credit risk investigations, or the treatment of collective investment undertakings (CIUs) and risks not in the model engines (RNIME)47 for market risk investigations). Overall, a large majority of the supervisory measures imposed in response to the findings of a TRIM investigation were already included in the first supervisory decision and not all institutions received a second one with reference to the same TRIM investigation.

Source: ECB.

46 Specifically, limitations restrict or modify the use of a model. This is in contrast to obligations, which

require remedial action without limiting the use of the model, and recommendations, which unlike obligations, are not legally binding. For further information on these terms, see the Guide to on-site inspections and internal model investigations on the ECB’s banking supervision website.

47 Used in market risk internal modelling to cover the risks which are not captured by the risk engines in the IMA for VaR, sVaR and IRC (for more detail, see the ECB guide to internal models).

Supervisory decisions were issued to enforce the remediation of cases of non-compliance with applicable regulatory requirements that were identified in the TRIM investigations



https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.guidetointernalmodels_consolidated_201910%7E97fd49fb08.en.pdf

Targeted Review of Internal Models – Observations and findings

27

4 Observations and findings

Following the completion of the TRIM on-site investigations and the general topics assessments, horizontal analyses were conducted by each centre of competence (general topics, credit risk, market risk and CCR). A summary of these analyses is provided in this section.

Overall, the outcomes of the TRIM investigations indicate that the internal models of SIs can generally continue to be used for the calculation of own funds requirements. However, there are a number of areas that require significant effort on the part of institutions to remediate findings in order to be fully compliant with regulatory requirements. While numerous findings have been raised across all risk types, it should be noted that in some cases these are also reflective of historic deviations in how requirements have been understood at national level, or of a lack of detailed guidance. In this respect, institutions should benefit in future from the detailed guidance provided by the ECB guide and by the relevant RTS and guidelines issued by the EBA, in particular as part of the regulatory review of the IRB approach.

Within this section, a number of the most significant observations and findings have been outlined. For the sake of conciseness, where practices were observed that are not compliant with regulatory requirements, no further details on how banks should address these issues have been included in this section. Such information is part of the supervisory follow-up process, an overview of which is included in Section 5. Also, a more detailed and comprehensive overview of the ECB’s understanding of applicable regulatory requirements related to topics within the scope of TRIM is provided in the ECB guide.

4.1 General topics

General topics are those that concern non-model-specific requirements that are part of the existing legal framework on internal models and are necessary to ensure an appropriate environment for the use of models. The individual sub-topics covered were:

• Overarching principles for internal models

• Roll-out and PPU

• Internal governance

• Internal validation

• Internal audit

• Model use

• Management of changes to the IRB approach

Most models are generally suitable for ongoing use by institutions, …

General topics cover the necessary infrastructure and governance requirements for an institution to run internal models for Pillar 1 capital purposes

…but there are a number of areas that still require significant improvement


28

• Third-party involvement

With the exception of “Overarching principles for internal models”, these sub-topics were considered in relation to credit risk models only. In this context, compared to the risk-type-specific assessments conducted in the form of on-site IMIs, the approach for general topics was a proportionate approach based on a documentation and questionnaire review coupled with targeted discussions with institutions during short on-site visits. For market risk and CCR models, the most relevant sub-topics (for example internal validation) were instead reviewed in the context of the corresponding on-site investigations.

4.1.1 Summary of findings

55 institutions were included in the horizontal analysis of general topics.48 In cases of outright non-compliance with current regulatory requirements, findings were raised and addressed by imposing certain obligations in the supervisory decisions addressed to the institutions, whereas deviations were addressed via recommendations and feedback letters, indicating that the practice of the institution was not in line with the principles expressed in the ECB guide. Based on the analysis of questionnaire information received from the institutions and the on-site visits, all 55 institutions received feedback letters (highlighting a total of 639 deviations), while 21 institutions also received supervisory decisions (raising a total of 45 findings).49

Figure 4 below provides a breakdown of the deviations and findings observed per sub-topic included in the general topics review. Most of the deviations refer to model use (132 deviations), internal governance (98 deviations), model changes (89 deviations) and internal validation (88 deviations). Regarding the findings, most deficiencies related to internal validation (14 findings), roll-out and PPU (11 findings) and management of model changes (seven findings).

48 These were significant institutions within the scope of TRIM employing internal models for credit risk,

excluding institutions that were already undergoing an on-site investigation on model governance or for which the first TRIM investigation was to take place in the second part of the project (in which case, general topics would be assessed as part of the on-site investigation).

49 After the “Right to be Heard” one decision has been dropped (which contained one finding), after the institution has provided evidence that allowed for the closure of the finding raised.

Most of the general topics findings were related to internal validation, roll-out and permanent partial use of models


29

Figure 4 Deviations and findings within general topics

Source: ECB supervisory information. Note: Deviations relating to “Identification of management body and senior management”, “General principles for internal validation” and “General principles for internal audit” were allocated to 3, 4 and 5, respectively.

Overall, the general topics review showed that most of the in-scope institutions had adequate practices with regard to fulfilling the broad scope of regulatory requirements concerning the “environment” for internal model use, but that generally these practices fell short of the upcoming standards under future regulation (notably the EBA RTS on assessment methodology for the IRB approach). Although it is not possible to demonstrate, it is understood that the qualitative requirements assessed in the general topics review are generally secondary drivers of RWA variability when compared with the modelling choices made by the institutions.

The most frequent deviations and findings are summarised below by sub-topic.

Overarching principles for internal models

Model risk management framework

Few institutions have a comprehensive framework for model risk management in place, and in the cases where there is one, it often requires improvement.

Documentation of internal models

Overarching principles or guidelines on model documentation are missing or incomplete for almost half of the institutions within the scope of the review.

Number of deviationsRecommendations and feedback letters

Number of findingsSupervisory decisions

Overarching principles1

Roll-out and PPU2

Internal governance3

Internal validation4

Internal audit5

Model use6

Management of model changes7

Third-party involvement8

132

54

88

98

70

71

89

37

14

4

11

0

4

2

7

3

Total Σ639 Σ45

https://eba.europa.eu/regulation-and-policy/credit-risk/regulatory-technical-standards-on-assessment-methodology-for-irb-approach

https://eba.europa.eu/regulation-and-policy/credit-risk/regulatory-technical-standards-on-assessment-methodology-for-irb-approach


30

Roll-out and permanent partial use

Compliance with PPU provisions

A majority of institutions did not have proper monitoring procedures for PPU exposures.

Practices in this regard varied across institutions, but in most cases there were no quantitative or qualitative thresholds in place to confirm the continued appropriateness of PPU.

Internal governance

Decision-making responsibilities

Institutions need to better formalise their decision-making process as well as the involvement of the management body/designated committee and senior management50 in the approval of material changes or other relevant aspects of the rating systems.

In several cases, the management body or senior management did not approve all relevant aspects of the rating systems or changes.

Understanding of the rating systems

A dedicated process was often not in place to ensure, maintain and improve the management body’s and senior management’s understanding of the rating systems.

Internal validation

Content of tasks of the validation function

Institutions often did not ensure that all appropriate validation analyses and adequate quantitative thresholds were in place, in particular for the back-testing, discriminatory power and stability tests and analyses of overrides.

Effective independence of the validation function

In some cases, institutions did not present an adequate organisational choice that ensured the independent allocation of the validation function within the institution.

50 The terms “management body” and “senior management” are defined in Article 3(1)(7) and (9) of the

Capital Requirements Directive (CRD IV). See Directive 2013/36/EU of the European Parliament and of the Council of 26 June 2013 on access to the activity of credit institutions and the prudential supervision of credit institutions and investment firms, amending Directive 2002/87/EC and repealing Directives 2006/48/EC and 2006/49/EC (OJ L 176, 27.6.2013, p. 338).

A majority of institutions need to improve their PPU monitoring and management

Institutions should have an organisational structure that ensures the independence of the validation function

http://data.europa.eu/eli/dir/2013/36/oj





31

Internal audit

Scope and frequency of review

Some institutions did not perform an appropriate or sufficiently frequent assessment or review of the rating systems and their operations to determine which rating systems required a more in-depth review.

Internal audit function

Some institutions lacked sufficient resources and/or staff with the necessary level of knowledge or skills.

Model use

Use test requirements

With regard to the role of the risk parameters in the relevant risk management processes, for two-thirds of the institutions, IRB parameters were not considered in certain internal processes for which institutions are encouraged to take them into account (especially with regard to collection and recovery policies and processes, and early warning systems).

Assignment of exposures

Several institutions did not present an appropriate process for treating unrated exposures or outdated ratings. In addition, the processes to define the maximum extent of overrides lacked completeness or were insufficient.

Management of changes to the IRB approach

Re-rating process

In most of the institutions, the re-rating process after a material model change or model extension was neither properly formalised nor covered in the relevant policy on the management of changes.

Classification

Some institutions lacked a formalised classification process to ensure that model changes and extensions were classified in a consistent way.


32

Third-party involvement

Independent performance monitoring

In some cases, institutions did not ensure proper monitoring of third-party performance in connection with outsourced tasks and/or the use of external data.

4.2 Credit risk – models for retail and SME portfolios

In total, 53 SIs with internal credit risk models were considered in part one of the TRIM execution phase, with 85 investigations carried out on credit risk models for retail and SME portfolios (ranging from one to four investigations per SI).

The review considered the risk parameters for internal credit risk models related to retail and SME portfolios, namely the PD, LGD and CCF. Owing to the nature of the portfolios under review in terms of limited off-balance sheet exposures, the detailed results presented in this report focus on the PD and LGD parameters, and not on the CCF parameter.

For most of the selected topics, a range of diverging practices with a potential impact on RWA variability was observed as well as cases of diverging understandings of regulatory requirements by institutions. Details on the findings that were raised for these models are provided in Section 4.2.2. In terms of ensuring a consistent interpretation of the requirements, significant progress has been made, and increased harmonisation of the implementation of the regulatory requirements should be observed when institutions have remediated the findings raised during the TRIM investigations.

4.2.1 Modelling landscape and key observations from the horizontal analysis

Each investigation reviewed the same set of predefined methodological aspects, which were identified in the preparatory phase of the TRIM project. An overview of the observed features of PD and LGD models in these areas (the “modelling landscape”) is provided below. Section 4.2.1.1 concerns observations specific to PD modelling and Section 4.2.1.2 concerns features of LGD modelling.51

51 These sections of the report are based on information collected from 79 (out of 85) on-site investigations

on models for retail and SME portfolios.

85 investigations were carried out on models for retail and SME portfolios


33

4.2.1.1 PD modelling landscape

General modelling approaches to the PD modelling

Three different types of PD model were observed, based on the granularity of the grade scale used (discrete or continuous) and how the PD estimates were calibrated (see Table 3).

Table 3 Types of PD model

Granularity of the scale

Continuous Discrete

Calibration is based on Long-run average at calibration segment level

Continuous direct estimates Discrete direct estimates

Long-run average at grade level

Grade-based estimation

Source: ECB.

11% of the PD models were based on a continuous granularity, where the PD estimates result from a transformation function that converts the score into a direct estimate of PD (continuous direct estimates)52. This approach may include an additional calibration step in order to achieve a calibration target.

In the remaining 89% of cases, a discrete scale was used. This form of PD calibration can be divided into two sub-cases: in 33% of PD models, a similar approach to continuous direct estimates was used, with the additional step of mapping the continuous PDs to a grade scale – either a master scale used across different portfolios or a grade scale specific to the portfolio (discrete direct estimates); in 56% of PD models, the PD used for capital requirements calculation was based on the long-run average default rate calculated at grade level (grade-based estimation).

Risk differentiation

Risk differentiation is aimed at identifying the relevant risk drivers and using them, on the basis of a chosen methodology, to rank or differentiate obligors or exposures into grades or pools according to their level of risk.

Selection of risk drivers considered in the model

While different practices were observed regarding the risk drivers considered in the model, most institutions used internal behavioural information (including past delinquency, which over 90% of the investigated internal models used as a driver in the scoring function), financial information and contract/obligor characteristics in their PD models. 52 See Article 169(3) of the CRR.

The vast majority of in-scope institutions use a discrete rating scale, combined in more than half of the cases with calibration at grade level

Most institutions include behavioural information, such as past delinquency, among the risk drivers


34

Homogeneity and heterogeneity analyses

Homogeneity within grades and heterogeneity across grades was one of the areas targeted during the TRIM investigations. In a majority of cases, no specific analyses were conducted by institutions, or the analyses were not considered appropriate. In order to provide clarity on this topic, the ECB further elaborated its understanding of the applicable requirements in Section 4.1.2 of the ECB guide.

Grade assignment dynamics

A dispersion of practices was observed in the TRIM investigations of retail and SME models regarding the dynamics of the grade assignment methods across different combinations of the following two extreme cases:

• the grade assignment method focused exclusively on the short term, resulting in an average PD at portfolio level that closely followed the yearly portfolio default rate;

• the grade assignment method focused exclusively on the long term, resulting in an average PD at portfolio level that was stable at the level of the long-run average portfolio default rate.

These differences have an impact on the average PD at portfolio level as well as on the RWA of the institutions.

In the light of this, as well as the observed lack of attention paid to this topic in many institutions, additional clarifications have been included in the ECB guide,53 reflecting the understanding that, although the time horizon used in PD estimation is one year, the rating/grade/pool assignment process should also adequately anticipate and reflect risk over a longer time horizon54 and take into account plausible changes in economic conditions.

Calculation of one-year default rates

Counting level

The counting unit for the default rate and the PD assignment level for the calculation of RWA are expected to be consistent, or if different levels are used, it should at least be shown that no biases are introduced in the estimation. As can be seen in Table 4, in the vast majority of cases there was consistency in the counting unit.

53 See paragraph 64 in the credit risk chapter of the ECB guide to internal models. 54 Ensuring an appropriate balance between drivers that are predictive only over a short-time horizon and

drivers that are more forward looking.

Different practices were observed regarding the grade assignment method for models related to retail and SME portfolios, and additional clarifications have been included in the ECB guide to increase awareness on this topic


35

Table 4 Counting level of the default rate

Default rate counting unit

Percentage of cases where PD assignment unit is consistent with

default rate unit

Percentage of cases where PD assignment unit is inconsistent with

default rate unit

Facility level 92.0% 8.0%

Obligor level 89.8% 10.2%

Group of economically dependent obligors 100.0% 0.0%

Source: ECB.

Most cases of inconsistency between counting units related to the use of a more aggregated level for PD assignment when compared with the counting unit for the default rate.

Long-run average default rate and calibration methodology

Institutions are expected to estimate PDs by obligor grade from the LRA DR.55 This requirement can lead to a number of areas of variability, in particular regarding the definition of the relevant period to capture the long-run average and the calibration methodology used by institutions.

Observation period used to calculate the LRA DR

From the TRIM horizontal analysis, different practices can be observed between (i) a majority of countries with no previous explicit guidance on this topic, which seems to have led institutions to use the available observed data only (i.e. in general, information since the inception of Basel II), and (ii) a minority of countries where there was explicit guidance on the time series to be used.

It was observed from this analysis that:

• in 86% of cases, the period used was at least 5 years;

• in 29% of cases, the period used was at least 10 years;

• in a majority of cases, the observation period used contained the years 2008 (70% of cases) and/or 2009 (90% of cases), which are generally considered downturn periods;

• in 66% of cases, only data after 2006 were used, while in 34% of cases information from earlier years was used.

55 See Article 180(1)(a) of the CRR.

Different practices were used to define the observation period for long-run average default rates

In most cases the years 2008 and/or 2009 were included in the observation period


36

Calculation of the LRA DR

Half of the institutions used the average of the observed default rates in the period considered to represent the long-run average, while the other half applied adjustments to obtain the long-run average of default rates. In most of the latter cases, the institutions did not have observed default data for the whole period of the likely range of variability of default rates and applied adjustments to obtain the LRA DR by:

• extrapolating the missing points in the time series;

• applying some kind of weighting to the observed default rates;

• using a combination of historical observation, forward-looking expertise and expert judgement.

Such adjustments may lead to unwarranted variability in the PD estimation if they are not duly justified.

Weighting approaches

The LRA DR is expected to be computed as the arithmetic average of the observed one-year default rates. However, in the retail exposure class, some discretion is possible where an increased weighting for more recent periods leads to a better prediction. In 66% of cases an arithmetic average was used in the LRA DR calculation, 5% used a higher weighting for more recent periods, 20% used a weighting based on the number of observations, and the remaining 9% used another form of weighting or calculation.

Calibration level

Institutions used calibration either at calibration segment level (44% of cases) or at grade level (56% of cases).

Margin of conservatism

When estimating risk parameters, institutions are expected to identify any deficiencies that may lead to a bias in their quantification. Institutions are required to address the identified deficiencies via appropriate adjustments and margins of conservatism (MoCs), including a margin for statistical uncertainty.56

Regarding the practices observed in TRIM, in around 40% of cases institutions were already applying an explicit MoC to their PD estimates, while in around 50% of cases institutions stated that an MoC was implicitly considered through conservative adjustments. In the remaining cases, no MoC was considered.

56 See Article 179(1)(f) of the CRR.

Long-run average default rates were not always calculated as the simple average of default rates, and adjustments were not always well justified

An explicit MoC was not always added to the PD estimates


37

In cases with an explicit MoC, margins were applied mostly to account for data and methodological deficiencies (62% of cases) and the general estimation error (71% of cases), while in 26% of cases they were applied to account for representativeness issues.

4.2.1.2 LGD modelling landscape

General modelling approaches to the LGD modelling

Component-based models

It should be noted that in a significant number of cases (around 80%), the LGD was estimated through a combination of different components – based on the split of the exposure between secured and unsecured, on the termination scenario, or on the use of intermediate phases in the recovery process (e.g. before and after entering into legal litigation) – which, when aggregated, result in an estimate of the LGD.

Continuous and discrete approaches

Depending on (i) the granularity of the grade scale used (continuous or discrete) and (ii) how the LGD estimates are calibrated (based on the long-run average at calibration segment level or at grade level), different model types were observed, as shown in Table 5 below:

Table 5 Types of LGD model

Granularity of the scale

Continuous Discrete

Calibration is based on Long-run average at calibration segment level

LGD direct continuous LGD direct discrete

Long-run average at grade level

Grade-based estimation

Source: ECB.

In approximately 45% of cases, the LGD estimates were using a continuous scale, either estimated directly or calculated as the aggregation of several components (LGD direct continuous).

In the remaining 55% of cases, a discrete scale was used. This can be divided into two sub-types based on the form of LGD calibration: in 45% of cases, the aggregation of several components resulted in a discrete scale (LGD direct discrete); in the remaining 10% of cases, the LGD estimates resulted from the long-run average calculated at grade level (grade-based estimation).

In a significant number of cases, the LGD was estimated through a combination of different model components

In 90% of cases, LGD estimates were direct estimates, using either a continuous scale (45%) or a discrete scale (45%)


38

Calculation of realised LGD

The concepts of economic loss and realised LGD are the basis for LGD estimation and any differences in the method of calculation may lead to significantly different LGD estimates. Therefore this was one of the areas selected for intense scrutiny during the TRIM investigations.

Level of calculation

In 78% of cases, the realised LGD was calculated at facility level, while in 11% of cases, it was calculated at obligor level. The remaining 11% were residual cases, such as when a mix of different assignment levels was considered (e.g. obligor for the unsecured part and collateral for the secured part) or the assignment level was represented by a group of connected obligors. The ECB guide clarifies the exceptional cases in which the LGD calculation at a more aggregated level than individual facility level is considered by the ECB to be compliant with applicable requirements.57

Recovery flows

Regarding the recovery flows allocation, it is worth noting that in around 40% of cases, recovery flows were not directly observed but calculated on the basis of the difference between exposure values on two consecutive dates or derived, even partially, from some other treatment. The ECB guide clarifies the criteria that should be met for this methodology to be considered eligible; in particular that all assumptions should be duly justified and clearly documented in order to adequately replicate the recovery flows that occur during the recovery process.58

Restructurings

Regarding the treatment of restructurings, in about one-third of cases, institutions were not able to connect the new facility after restructuring to the facility prior to restructuring. In addition, in a significant number of cases (71%), institutions did not consider in the definition of loss the diminished financial obligation that can result from the restructuring owing to changes in the payment plan and the consequent delays in cash flows, grace periods, changes in interest rates, etc.

Discounting of cash flows

A wide variety of practices were observed in the discounting of cash flows, i.e. the use of a fixed rate (30%), the contractual rate (19%) or a base rate with (19%) or without (10%) an additional risk premium. This is an area in which the EBA has provided

57 See paragraph 99 in the credit risk chapter of the ECB guide to internal models. 58 See paragraph 100(c) in the credit risk chapter of the ECB guide to internal models.

The methods used to calculate realised LGD were subject to intense scrutiny during the TRIM investigations as potential drivers of significant unwarranted RWA variability


39

guidance on a single approach that institutions are expected to follow (generally the 3-month EURIBOR increased by a 5 percentage point add-on).59

Treatment of multiple defaults

Based on the practices observed, the different treatments of multiple defaults were clustered in the following categories:

• in 34% of the models, there was no treatment of multiple defaults;

• in 26% of models, the “calendar year approach”, in which two consecutive defaults of the same exposure are considered as one if the two defaults occur within 12 months or within the same calendar year, was applied;

• in 40% of models, a methodology was applied in which the time between defaults is counted from the return to normal of the first default.

For exposures that return to non-defaulted status, in 50% of cases institutions did not compute realised LGD for cured cases but instead made assumptions regarding the realised losses (e.g. a zero loss or the consideration of indirect costs only).

Additional drawings

Regarding additional (after default) drawings, 40% of retail models followed an adequate approach for the treatment of additional drawings (i.e. alignment between the EAD considered for CCF purposes and the EAD considered in the denominator of realised LGD), while in 20% of retail models additional drawings were not possible for the specific type of exposure. In the remaining 40% of cases, the approach did not ensure alignment between the EAD considered for CCF purposes and the EAD considered in the denominator of realised LGD, leading to an inconsistent approach.

In the case of non-retail exposures, in the vast majority of cases (85%) there was inappropriate treatment, since none of them considered the additional drawings at the same time both in the numerator and denominator of realised LGD and in the definition of exposure considered for the purposes of CCF estimation. In the remaining 15% of cases, additional drawings were not possible for this type of exposure.

Repossessed but not yet sold collateral

Regarding the treatment of repossessed but not yet sold collateral, more than half of the institutions used repossession as a recovery tool, of which a majority applied a haircut to the value of repossessed collateral (42% of total cases), while others (11% of total cases) took the value of repossessed collateral directly (i.e. no haircut was

59 See paragraph 143 of the EBA Guidelines on PD estimation, LGD estimation and the treatment of

defaulted exposures (EBA/GL/2017/16).

https://eba.europa.eu/sites/default/documents/files/documents/10180/2192133/f5a2e068-dc4b-4a0e-a10f-378b517ac19c/Guidelines%20on%20PD%20and%20LGD%20estimation%20(EBA-GL-2017-16)_EN.pdf

https://eba.europa.eu/sites/default/documents/files/documents/10180/2192133/f5a2e068-dc4b-4a0e-a10f-378b517ac19c/Guidelines%20on%20PD%20and%20LGD%20estimation%20(EBA-GL-2017-16)_EN.pdf


40

applied, which is generally not acceptable)60. In 47% of cases, repossession was not used as a recovery tool (owing to the legal framework) or the cash flow was only recognised once the collateral was sold.


While different practices were observed regarding the risk drivers considered in the model, most institutions used contract characteristics (including type of collateral and type of products) in their LGD models. Obligor characteristics and, in the case of retail exposures, internal behavioural information (i.e. information on the delinquency of the obligor) were the other types of risk driver most commonly used (in around 35% and 20% of cases, respectively).

In the case of models in which an LGD facility scale is used for the purpose of calculating regulatory capital requirements, statistical analyses aimed at ensuring the homogeneity/heterogeneity of the grades were performed when developing the model in 47% of cases, while in 35% of cases institutions relied solely on expert judgement. In 18% of cases no analysis was conducted. Regarding regular monitoring of models, the situation is worse and the vast majority (74%) of institutions did not have a process in place to check the homogeneity/heterogeneity of grades.

Treatment of incomplete recovery processes

Although the CRR requires the use of all defaults for the purpose of LGD estimation,61 wide variations were seen in the treatment of incomplete recovery processes.

The treatment of open recovery processes encompasses (i) the definition of a period of time after which the recovery process should be considered as closed for the purpose of LGD estimation (time-to-workout) and (ii) the development of an appropriate treatment to incorporate the relevant information from incomplete recovery processes (i.e. the ones which have not yet reached the time-to-workout) in the LGD estimates.

Time-to-workout

In 59% of cases, institutions already had a definition of time-to-workout. The length of this period varied from 3 to 15 years, depending on the country, the exposure class and the recovery processes. The definition of time-to-workout is strongly linked to the idiosyncrasies of the institution’s recovery process, which can justify differences between institutions in the same country and for the same type of exposure. However, the longer the time-to-workout, the more challenging will be the modelling of

60 See paragraph 117 of the EBA Guidelines on PD estimation, LGD estimation and the treatment of

defaulted exposures (EBA/GL/2017/16). 61 See Article 181(1)(a) of the CRR.

Most institutions use contract characteristics in their LGD models

The treatment of incomplete recovery processes showed wide variations


41

incomplete recovery processes, since there is less information available for the later stages of the recovery process.


It is important to highlight that 32% of the institutions did not explicitly take into consideration incomplete recovery files, while almost 68% of the institutions already considered incomplete files in the LGD estimation, either by extrapolating cash flows (46.5%) or by assuming no further recoveries (21.5%)62. Looking at institutions which had already implemented a methodology to extrapolate cash flows, a quite diverse impact of these methodologies can be observed – a wide spectrum of results across the range of what would be the LGD considering full recovery for the incomplete recovery processes and the LGD considering zero additional recovery for incomplete recovery processes.

Long-run average LGD


Regarding the level of calculation of the long-run average LGD (LRA LGD), in a majority of cases, institutions calculated it at either portfolio/calibration segment (40%) or grade/pool (24%) level.63 The remaining 36% of models assessed had an inappropriate approach, either because an LRA LGD was not calculated (14%) or because it was computed only at the level of each component (22%).

Observation period used for the LRA LGD

In terms of the time frame considered, and although a common period was used by most institutions between 2008 and 2013 (in 78% of cases, all six of these years were included as part of the LRA period), there were still significant differences in the observation period used across institutions, as well as differences in the weighting approach.

Weighting approach

While in a majority of cases (60%), institutions used the default weighted average as required by the CRR64, the remaining institutions used an inappropriate weighting approach – around 20% used EAD weighting and 14% did not compute an LRA.

62 This approach is normally linked with the definition of the minimum period of the recovery process in

order for defaults to be considered for the purpose of LGD estimation. 63 For the purpose of this analysis, the latter includes cases of continuous models which use intervals of

LGD values in the calibration of the LRA. 64 See Article 181(1)(a) of the CRR.

A majority of institutions calculated the LRA LGD at either portfolio/calibration segment or grade/pool level

In a majority of cases, the years 2008-2013 were included in the observation period to calculate the LRA LGD


42

Downturn LGD

In 53% of cases, an explicit period was defined to derive an LGD that is appropriate for an economic downturn. In the remaining cases, either the institution applied an adjustment without identifying a downturn period or no adjustment was applied.

For the institutions identifying a downturn period, although similar economic downturn periods for the same country and exposure type were expected, significant differences were still observed, not only in the period identified but also in the duration. In 76% of cases, the downturn period included the years 2008 or 2009, while in 26% of cases, the period included data from before 2006.

Similarly, a wide variety of practices were observed in relation to the type of adjustment. In 41% of cases, the downturn adjustment was based on observed values during the selected downturn period. In 26% of cases, the downturn adjustment was obtained by stressing model components (e.g. interest rates, collateral values, etc.), while in 11% of cases, no downturn adjustment was applied.


When estimating risk parameters, institutions should identify any deficiencies that may lead to a bias in their quantification. Institutions are required to address the identified deficiencies via appropriate adjustments and MoCs, including a margin for statistical uncertainty.65

Regarding the practices observed in TRIM, in around 50% of cases, institutions were already applying an explicit MoC, while in around 40% of cases, institutions stated that MoC was implicitly considered through conservative adjustments. For the remaining cases, no MoC was considered. Regarding the cases with an explicit MoC, the triggers were data and methodological deficiencies (74% of cases), representativeness issues (80%) and general estimation error (80%).

Expected loss best estimate and LGD in-default

A variety of different practices were seen for calculating LGD in-default, including using the sum of the expected loss best estimate (ELBE) plus an add-on, and “direct” LGD in-default approaches using dedicated models.

Regarding the estimation of ELBE, in 21% of cases, there was no specific estimation in place (i.e. the non-defaulted (downturn) LGD continued to be used when the exposure was in default), and in 25% of cases, the ELBE estimate was based on or equal to the credit risk adjustments. In the remaining 54% of cases, the ELBE was instead evaluated using a dedicated model.

65 See Article 179(1)(f) of the CRR.

Significant differences were observed in how institutions defined a downturn adjustment

As in the case of PD, LGD estimates do not always include an MoC

Various practices were observed for calculating LGD in-default, including using the sum of ELBE plus an add-on, and “direct” LGD in-default approaches using dedicated models


43

Regarding the estimation of LGD in-default, in 29% of the investigations, there was no specific estimation in place (i.e. the non-defaulted (downturn) LGD continued to be used when the exposure was in default) and the exposure status was not reflected. Regarding cases with a specific estimation of LGD in-default, in 33% of the investigations, the LGD in-default was calculated as the sum of the ELBE plus an add-on, while in 38% there was a “direct” LGD in-default approach.


In total, 2,000 findings from the TRIM investigations relating to models for retail and SME portfolios were included in the horizontal analysis (covering 85 investigations), which includes additional findings identified as a result of consistency checks and horizontal analyses conducted on the assessment reports delivered by the inspection teams. In terms of severities attributed to these findings, approximately 24% and 46% were classified as F1 and F2, respectively, while 25% and 5% of the findings were classified as F3 and F4, respectively. Around 7% of the findings were not related to any binding regulatory requirements; for example, they may have been raised only in reference to draft technical standards or guidelines not already in force. The average number of findings per investigation was 24, while the maximum was slightly over 50.66

Figure 5 shows the distribution of findings per modelling area and by severity.

66 Findings related to the reviews of general topics (see Section 4.1.1) and data quality (see

Section 4.2.2.3) should be considered in addition to these figures to the extent that they refer to the same credit risk models.

Investigations of models for retail and SME portfolios generated 2,000 findings


44

Figure 5 Overview of findings for credit risk models related to retail and SME portfolios

Source: ECB supervisory information. 1) This column provides information on the frequency of occurrence of each topic in the TRIM assessment reports considered in this analysis. One assessment report may include several findings within the same topic. 2) Owing to the scope of the portfolios under review in terms of reduced volume of off-balance sheet exposures, the CCF-related findings are not considered representative for horizontal analysis. 3) “Overarching” includes investigations in connection with requests for material model changes which had a wider scope than regular TRIM investigations.

Findings were raised across all of the in-scope risk parameters (PD, LGD and CCF) as well as for overarching areas.

The most common findings from the TRIM investigations of credit risk models for retail and SME portfolios are summarised below. As indicated in Section 4.2 owing to the scope of the portfolios under review in terms of reduced volume of off-balance sheet exposures, this section focuses on PD and LGD-related findings.

Percentage of invest-igations

with findings1Distribution of findings by severityTopic

Number of

findings


with F3/F4 findings

Number of

investiga-tions

574 85

1,046 79

163 64

217 85

CC

F 2

LGD

PD

140 77 217

12 14 26

102 12

17 4 21

75 12

15 5 20

32 4 36

35 1 36

64 47 111

47 33 80

51 28 79

55 24 79

42 29 71

64 32 96

59 21 80

73 43 116

267 67 334

54 37 91

57 35 92

61 33 94

70 13 83

24 8 32

141 41 182

F1-F2 F3-F4


Grade assignment dynamicsCalculation of one-year default rateLong-run average default rate


Framework for review of estimatesCalculation of realised LGD


RDS completeness



Downturn adjustment


ELBE and LGD in-default

Framework for review of estimates

Realised CCF


Long-run average

Downturn adjustment



Completeness of the RDS

Overarching 3

86% 36%

33% 6%

62% 15%

71% 29%

80% 32%

75% 36%

100% 53%

87% 42%

61% 23%

87% 34%

67% 30%

86% 29%

78% 30%

78% 38%

81% 47%

38% 2%

44% 6%

25% 8%

17% 8%

28% 3%

17% 3%

30% 20%

72% 38%


45

4.2.2.1 PD-related findings


In more than half of the models reviewed, findings were raised concerning low risk differentiation of PD models owing to the low discriminatory power of the scoring/ranking functions and/or as a result of inappropriate homogeneity/heterogeneity of the grades. In a significant number of cases, these findings were assigned a high severity (at least F3).

Findings were also frequently triggered by the inappropriate justification of modelling assumptions and issues with the range of application of the models.


Findings were mainly related to the lack of analysis of the grade assignment dynamics, and its implications in terms of risk quantification and parameter estimation.

Calculation of one-year default rate

Findings were mainly related to:

• shortcomings in the method used to compute the default rate (e.g. identification of obligors or facilities not accounted for in the numerator, identification of obligors or facilities not accounted for in the denominator, absence of analysis or accounting related to the bias resulting from considering several credit facilities of the same client, non-calculation of the one-year default rate for the subset of obligors that did not have a rating at the start of the relevant observation period but were in the range of application of the model);

• the lack of representativeness of the definition of default across time.


In around 40% of the models reviewed, findings were raised concerning the period used to define the long-run average not being properly justified or appropriate. In a significant number of cases, these findings had a high severity. Similar percentages were observed for findings regarding the lack of, or inappropriate, justification of the calibration methodology; for example, where the institution used weighting in the calculation of the LRA DR.

Many of the findings related to PD concern shortcomings in risk differentiation


46


Most of the findings concerned the inappropriate calculation of the level of conservatism and the absence of a framework for the identification of deficiencies and quantification of the respective MoC. In a significant number of cases, and for both reasons, these findings had a high severity.

Framework for the review of estimates

A majority of findings related to an incomplete framework. In a significant number of cases, these findings had a high severity.

4.2.2.2 LGD-related findings


Issues regarding the calculation of realised LGD were identified in all of the investigations. In particular, these findings related to the lack of the necessary information to compute realised LGD, the definition of economic loss not being comprehensive enough and the process of allocation of recoveries and costs leading to bias in LGD estimates. In a significant number of cases, these findings were assigned a high severity.


A majority of investigations identified deficiencies with regard to the risk drivers for LGD models, in particular in relation to missing or irrelevant risk drivers. Poor risk differentiation of the LGD models was an issue in almost half of the investigations, owing to low discriminatory power of the scoring/ranking functions and/or inappropriate homogeneity/heterogeneity of the grades.

Reference dataset completeness

Most findings related to missing information in the reference dataset67 to estimate LGD and exclusions from the reference dataset not being properly justified. In the former case, findings were raised for around 40% of the investigations.

67 The reference dataset is the dataset provided by the institution to be tested by the assessment team

during the execution of the credit risk on-site investigations.

The calculation of realised LGD was the topic that generated the most findings


47


The main findings were related to non-consideration of incomplete cases in the LGD estimation and the inadequate definition of the time-to-workout.


The most significant findings were related to the use of a weighting other than facility-weighted average and to the calibration methodology of the long-run average, including lack of or inappropriate adjustments to ensure a representative long-run average.

Downturn LGD

A majority of findings with regard to the downturn LGD related to the identification of the relevant downturn conditions, including cases where the institution did not characterise an economic downturn in terms of economic and credit indicators or where the institution did not take into account a sufficiently long historical dataset of such indicators. In around one-third of the models, issues with the quantification of the downturn adjustment were identified (including cases where no downturn adjustment was applied).


Most of the findings on this topic concerned the inappropriate calculation of the level of conservatism and the absence of a framework for the identification of deficiencies and quantification of the respective MoC.


In around 20% of cases, findings were raised concerning the lack of dedicated ELBE or LGD in-default models, while in around one-third of cases, findings were raised owing to weaknesses in the modelling approach.


A majority of findings in this sub-topic related to a lack of relevant analysis (affecting almost half of the models); for example, the institution not performing the minimum scope of tests expected or not having an appropriate set of metrics to test model performance.


48

4.2.2.3 Data quality findings

Data quality reviews were a core part of the investigations of credit risk models for retail and SME portfolios. Two different levels of investigation intensity were available in the assessment of data quality, of which the higher intensity investigations (around 20% of investigations) allowed a deeper analysis and more proportionate assessment during the data quality review.

In total, there were 445 data quality findings from the TRIM investigations related to models for retail and SME portfolios that were included in the horizontal analysis (85 investigations). The average number of data quality findings per investigation was five, while the maximum number of findings per investigation was 15. The findings concerned 505 identified shortcomings.68 In terms of severities attributed to these findings, approximately 21% and 56% were classified as F1 and F2, respectively, while 22% and 1% of the findings were classified as F3 and F4, respectively.

Figure 6 shows the distribution of shortcomings per data quality topic and by severity.

Figure 6 Overview of data quality shortcomings

Source: ECB supervisory information.

As seen in Figure 6 findings for this topic can be categorised under the institution’s technical implementation of the definition of default or in relation to data maintenance. While findings were identified and recorded as part of TRIM, in general the appropriateness of the definition of default and of its implementation was or will be reviewed in the context of the implementation of the new definition of default pursuant to the relevant EBA guidelines69 and of the ECB Regulation related to the materiality threshold70.

68 As the individual data quality-related shortcomings were grouped differently across investigations, the

findings that capture several deficiencies have been disentangled with the aim of making them comparable across on-site investigations. For example, a single finding in an assessment report may capture shortcomings both in the technical implementation and in the controls in place, i.e. two shortcomings.

69 See EBA Guidelines on the application of the definition of default under Article 178 of Regulation (EU) No 575/2013 (EBA/GL/2016/07).

70 See Regulation (EU) 2018/1845 of the European Central Bank of 21 November 2018 on the exercise of the discretion under Article 178(2)(d) of Regulation (EU) No 575/2013 in relation to the threshold for assessing the materiality of credit obligations past due (ECB/2018/26) (OJ L 299, 26.11.2018, p. 55).


with short-comings

Distribution of shortcomings by severityTopic

Number of short-comings


with F3/F4 short-

comings

Number of

investiga-tions

130

85

Dat

a m

aint

enan

ce

53 18 71

152 54 206

80 18 98

76 18 94

27 9 36

F1-F2 F3-F4

Regulatory compliance

Technical implementationIT infrastructure – docu-mentation and testing Data management anddata quality processesOutcomes of TRIM technical tests

32% 9%

62% 15%

62% 18%

92% 41%

55% 18%

Def

initi

on o

f de

faul

t

375

Data quality reviews were an important part of the credit risk investigations, with enhanced focus compared to previous internal model reviews

There were more than 400 data quality findings from the investigation of models for retail and SME portfolios

https://eba.europa.eu/sites/default/documents/files/documents/10180/1721448/052c260f-da9a-4c86-8f0a-09a1d8ae56e7/Guidelines%20on%20default%20definition%20(EBA-GL-2016-07)_EN.pdf

https://eba.europa.eu/sites/default/documents/files/documents/10180/1721448/052c260f-da9a-4c86-8f0a-09a1d8ae56e7/Guidelines%20on%20default%20definition%20(EBA-GL-2016-07)_EN.pdf





49

Shortcomings around data management and data quality processes were identified in almost all on-site investigations (92% of investigations), and were often of high severity (41% of investigations having either an F3 or F4 finding on this topic).

Within data management and data quality processes, the most prominent shortcomings across the different on-site investigations related to the current data quality system, procedures and processes (76% of TRIM on-site investigations presented shortcomings in this area). These included, for example, the use of metrics for data quality monitoring and mechanisms for the identification, remediation and follow-up of data quality issues. There were also widespread findings relating to the control framework in place (62% of on-site investigations). Other shortcomings related to weaknesses of the data quality framework (59% of on-site investigations) and weaknesses in the allocation of roles and responsibilities relating to data management, leading to a lack of accountability (59% of on-site investigations).

4.3 Credit risk – models for low-default portfolios

In total, 76 investigations were carried out on the selected credit risk models for LDPs, covering 48 SIs (ranging from one to three investigations per SI).

The credit risk models reviewed in this part of the project related to the following exposure classes: corporates – other, corporates – specialised lending, and institutions.71 Similar to the models for retail and SME portfolios, the selection of LDP models for review was primarily based on an assessment of the materiality and criticality of the model considered.

The review considered the risk parameters for internal credit risk models, namely the PD, LGD and CCF. The detailed results presented in this section of the report focus on the PD and LGD parameters, as these were the parameters most frequently assessed in the LDP investigations. The TRIM investigations also included a review of data management practices applied to the credit risk models under review, as well as an assessment of the quality of current and historical data used for IRB modelling purposes.

For many of the topics assessed, a number of diverging practices possibly leading to unwarranted RWA variability were observed, as highlighted in the corresponding findings and the key observations from the horizontal analysis. However, and as mentioned in relation to the review of models for retail and SME portfolios, TRIM has also enabled significant progress to be made towards a more consistent understanding of regulatory requirements, with guidance being provided in the ECB guide, which complements the EBA guidelines and RTS issued as part of the regulatory review of the IRB approach. This common understanding will also facilitate increased harmonisation of the implementation of regulatory requirements once

71 As indicated in Section 3.2.3, the term “low-default portfolio” was used in TRIM for simplicity. Some

models assigned to the LDP category were not strictly LDP models, while others assigned to the “retail and SME” category had the characteristics of LDP models.

Many high severity shortcomings were identified around data management and data quality processes

The 76 investigations on credit risk models for LDPs covered models for exposures to corporates – other, corporates – specialised lending, and institutions

Diverging practices with a potential impact on RWA variability were observed for many topics


50

institutions have remediated the findings raised during the TRIM investigations. This, in turn, will contribute to a reduction of the non-risk-based RWA variability.

Observations from the horizontal analysis of the LDP modelling landscape are provided in Section 4.3.1, and details on the findings raised for these models are provided in Section 4.3.2.


A horizontal analysis was performed based on information collected from the assessment reports for the LDP investigations with the aim of understanding and comparing the most relevant model features. The main observations from the horizontal analysis are provided for PD parameters in Section 4.3.1.1 and for LGD parameters in Section 4.3.1.2.72

4.3.1.1 PD modelling landscape

General modelling approaches to the PD modelling

Regarding the granularity of the grade scale used, it was observed that in 92.5% of cases a discrete scale was used to determine the final PD estimates. In the remaining 7.5% of cases, the models were based on continuous rating scales, where the PD estimates result from a transformation function that converts the score into a direct estimate of PD. This approach may include an additional calibration step in order to achieve a calibration target, which potentially leads to adjustments of PDs.

The 92.5% of models that used a discrete rating scale can be divided further: in approximately 36% of cases a model-specific rating scale was used (where the PD estimates are specific to the rating system) and in 56% a master scale approach was used (where the PD estimates are determined via a common rating scale for several rating systems at the level of the institution).

72 The number of assessment reports considered for each parameter differs, as not all institutions

developed their own LGD models. The horizontal analysis relating to the PD parameter was based on information collected from 74 assessment reports covering 80 PD models and representing 75 of the 76 in-scope LDP investigations (one assessment report focused only on CCF and thus was not relevant for the horizontal analysis of the PD parameter, while another report encompassed two investigations). The horizontal analysis relating to LGD was based on 49 LDP investigations, covering 50 LGD models.

Just over 90% of models used a discrete scale for PD estimation (rather than a continuous direct estimate of PD)


51


Approaches used for risk differentiation

To differentiate the risk of obligors, some institutions used analytical risk differentiation functions for ranking purposes, whereas others relied more extensively on an expert-based differentiation process. A number of approaches were observed in the horizontal analysis.

• In 27% of cases institutions had developed a risk differentiation function that considered the default event as the target variable.

• A shadow rating modelling approach was used in 26% of cases. This approach aims to identify the main factors that explain external ratings provided by an external credit assessment institution or similar organisation, rather than predicting directly observed defaults.

• In 23% of cases an internal rating (e.g. computed by means of expert judgement) was used as the target variable of the risk differentiation function.

• For the remaining cases, there were a number of different approaches, the most common of which was to use risk differentiation functions based entirely on expert judgement (16% of cases). Other observed approaches include the use of extended definitions of default (institutions adopting a wider internal definition of default for the purpose of developing the risk differentiation function), expert-based rating assignment processes, and models simulating defaults.

Selection of risk drivers to be considered in the model

A wide variety of risk drivers were observed in relation to ranking obligors. The horizontal analysis showed that qualitative and financial information was commonly considered in the models for the three exposure classes considered in LDPs (typically in over 80% of cases), while geographical information and external risk assessments were less commonly used.

Homogeneity and heterogeneity analyses

In the context of LDPs it was observed that institutions had not performed homogeneity/heterogeneity analyses. Also, it was difficult to obtain meaningful results from classical statistical analyses owing to the scarcity of data underlying the portfolios under investigation. Nonetheless, the horizontal analysis indicated cases where institutions should more properly justify the number of grades used, in particular where some grades are sparsely populated. These issues are due to the combination of a high number of grades and a low number of obligors/ defaults, which makes it very difficult to demonstrate that the PD scales enable an appropriate risk differentiation.

Various approaches to risk differentiation were observed, including shadow rating modelling to address the problem of data scarcity


52


The focus of the horizontal analysis of grade assignment dynamics was to understand the institutions’ practices with regard to (i) the time horizon considered when assigning obligors to grades or pools, and (ii) the effect that the grade assignment dynamics can have on PD quantification, the monitoring of risk (via discriminatory power and predictive ability analysis) and the volatility of capital requirements.

In relation to the time horizon, and in line with the ECB guide, the assignment process should adequately anticipate and reflect risk over a time horizon longer than one year and take into account plausible changes in economic conditions. However the horizontal analysis showed that in 41% of cases institutions adopted a time horizon of one year and in 24% of cases institutions did not explicitly define a time horizon. In the remaining 35% of cases a time horizon greater than one year was used.

Regarding the relationship between the grade assignment dynamics and the chosen method for the PD risk quantification, only 39% of institutions validated the adequacy of the PD quantification method by explicitly considering the underlying grade assignment dynamics. Finally, only in a very small number of cases could a specific assessment of the impacts of the grade assignment dynamics on the actual dynamics and volatility of capital requirements be observed.

Use of ratings of third parties

In 92% of cases institutions used the ratings of third parties in the rating assignment process. Of these:

• 46% used a bottom-up approach where a standalone rating of the obligor is determined first before the effect of the third-party’s rating is introduced;

• 9% employed a top-down approach where the strength of the influence/support of the third party is considered at the beginning of the rating assignment process;

• the remaining 45% used a mixed approach where the evaluation of the strength of the relationship between the obligor and the third party can lead to the rating of the third party to being directly assigned to the obligor, without calculating the initial rating assigned to the obligor.

It is worth highlighting that, in contrast to the bottom-up approach, in top-down and mixed approaches there is no evaluation of the obligor’s riskiness at the start of the rating assignment process (i.e. an initial standalone rating is not assigned to the obligor before the effect of the third party’s rating is introduced). Irrespective of the approach chosen, institutions are expected to provide proper justifications demonstrating that all the risk characteristics of the obligor have been properly taken into account in the final rating.

A variety of practices was observed when analysing the main elements used to define the strength of the support relationship between a third party (either private or public) and an obligor, such as the existence of guarantees or the third party’s strategic

In over 90% of cases institutions used third party ratings in the rating assignment process


53

importance to the group. In over three-quarters of cases intensity scales were used as instruments to determine the strength of the link between a third party and an obligor. In all the cases where procedures for evaluating the support relationship between third party and obligor were in place (e.g. a questionnaire), it was observed that these procedures mainly relied on expert judgement and not on data-driven processes.

Treatment of joint credit obligations

The horizontal analysis highlighted diverging practices for situations where an institution has a single exposure to several entities. In particular there was a lack of clarity about the difference between the treatment of joint obligors and the treatment of groups of connected clients. In addition, deficiencies were identified in the treatment of joint obligors in the calculation of realised default rates. This resulted in the development of a common ECB understanding73 on this topic. In particular, institutions should consider a joint credit obligation as an exposure to two or more entities that are equally responsible for the repayment of the credit obligation. This notion does not extend to a credit obligation of an individual obligor secured by another individual or entity in the form of a guarantee or other credit protection. The group of entities jointly obliged on a certain exposure may be indicated as one single joint obligor. In accordance with Article 178(1) of the CRR, a default shall be considered to have occurred with regard to a particular obligor when either or both of the conditions included in that provision have taken place. It is the ECB’s understanding that to comply with Article 178(1) of the CRR, and considering Articles 4(1)(78) and 180(1) of the CRR, whenever the definition of default is applied at obligor level, a joint obligor should be considered as a separate obligor and the default on a joint credit obligation should be counted separately from the default of individual obligors. Therefore, each obligor74 must be counted separately in the calculation of the one-year default rate, for both estimation and validation purposes.

Overrides analysis

Overrides of model outputs were typically (in 97% of cases) applied at the final step of the rating assignment process to derive the final rating. Additionally, overrides of an intermediate rating (e.g. financial score) were also observed in 5% of the methodologies analysed.

The horizontal analysis of overrides indicated a number of areas for improvement. 35% of institutions did not have a dedicated framework for monitoring overrides. 53% of institutions did not define a maximum acceptable rate of overrides, and a similar proportion had not defined a limited scale of non-conservative overrides. Finally, 32% of institutions did not monitor the performance of the model before and after overrides.

73 The ECB’s understanding of the treatment of joint credit obligations for exposures other than retail could

be further developed in line with future regulatory developments. 74 Intended as either individually or jointly (and severally) exposed to an institution.

The horizontal analysis of overrides indicated a number of areas for improvement


54

Risk quantification

Approaches used for risk quantification

For the purposes of horizontal analysis, the PD models can be classified according to the type of data used: internal, external or pooled. It was observed that in 31% of cases only internal data were used for risk quantification purposes, in 29% of cases only external data were used, in 16% of cases a combination of internal and external data were used, in 10% of cases pooled data were used, and in the remaining cases an expert-based approach was applied (without observing default rates).

In the instances where external or pooled data were used for risk quantification, it was noted that in 57% of cases there was either no analysis of the representativeness of external/pooled data or the analyses conducted were incomplete or insufficient to draw any reliable conclusions on the representativeness of the data used. In addition, only 50% of institutions performed an assessment of the consistency of the definition of default applied to the external or pooled data with their internal definition of default.

Period used for the calculation of the long-run average default rate

A number of practices were observed with respect to the periods used by institutions to define the likely range of variability of one-year default rates. In the context of LDP models, the period considered as representative of the likely range of variability of one-year default rates was in many cases constrained by data availability and the representativeness of data. A significant number of institutions were able to use data from the mid-2000s, likely corresponding to the beginning of data storage under the Basel II standards. For the models related to portfolios with very few defaults (typically exposures to institutions or very large corporates), the most common practice was to use all available external data at the time of the calibration without conducting any further analysis. In addition, it was detected that there were several models that had not been recalibrated after the financial crisis of 2008-09; the lack of model recalibration for such an extended period of time could be considered questionable unless this was justified by proper validation of the estimates.


In around 23% of cases institutions applied an explicit MoC to their PD estimates at grade or portfolio/calibration level, while in around 57% of cases institutions stated that an MoC was implicitly considered through conservative assumptions. For the remaining cases, no MoC was considered.

Where an explicit MoC was considered, in 82% of cases it was applied to account for data and methodological deficiencies, in 24% of cases to account for

A shortage of internal data was one of the reasons identified for using external information for the calculation of the long-run average default rate


55

representativeness uncertainty and in 82% of cases to account for general estimation error, in line with the EBA Guidelines on PD and LGD estimation75.

4.3.1.2 LGD modelling landscape

General modelling approaches to the LGD modelling

Component-based models

With regard to the structure of the LGD models, 74% of the models estimated LGD using a component-based approach. Of these, the majority (70%) used an approach based on the split of the exposure between secured and unsecured components.

Continuous versus discrete approaches

Depending on (i) the granularity of the grade scale used (continuous or discrete) and (ii) how the LGD estimates are calibrated (based on the long-run average at calibration segment level or at grade level), different model types were observed.

In approximately 62% of cases, the LGD estimates result from a continuous scale, either estimated directly or calculated as the aggregation of several components (direct, continuous LGD).

In the remaining 38% of cases a discrete scale was used. This can be divided further: in 18% of cases the aggregation of several components resulted in a discrete scale (LGD direct discrete), and in 20% of cases the LGD estimates resulted from the long-run average calculated at grade level (grade-based estimation).


As part of the horizontal analysis a distinction was made between institutions that estimated LGD based on calculated realised LGD and those which used realised LGD for back-testing or validation purposes only. In over half of the investigations (54%) it was reported that LGD estimation was not based on realised LGD, although in the majority of these cases (67%), realised losses were calculated for back-testing purposes. In the context of the LDP models reviewed it should be noted that 63% of the models, for which realised LGD was not calculated for estimation purposes, are expected to be decommissioned in the light of the implementation of the Basel III framework through the amendments to the CRR.

75 EBA Guidelines on PD estimation, LGD estimation and the treatment of defaulted exposures

(EBA/GL/2017/16).

74% of the LGD models used a component-based approach, typically based on the split of the exposure between secured and unsecured

Around 40% of models used a discrete scale for LGD estimation

https://www.eba.europa.eu/sites/default/documents/files/documents/10180/2033363/6b062012-45d6-4655-af04-801d26493ed0/Guidelines%20on%20PD%20and%20LGD%20estimation%20%28EBA-GL-2017-16%29.pdf?retry=1

https://www.eba.europa.eu/sites/default/documents/files/documents/10180/2033363/6b062012-45d6-4655-af04-801d26493ed0/Guidelines%20on%20PD%20and%20LGD%20estimation%20%28EBA-GL-2017-16%29.pdf?retry=1


56

Recovery flows

In a majority of cases (65%), institutions were able to directly identify cash flows without a calculation/estimation step. In 13% of cases an implicit calculation was used where recovery flows were based on the difference between exposure values at two dates. The remaining cases (22%) employed a mixed approach where, for example, the recovery cash flows were directly identifiable for one component but were computed based on the difference in EAD for other components.

Discounting of cash flows

A wide variety of practices were observed in the discounting of cash flows. Only 4% of the models considered a discount rate in line with EBA Guidelines on PD and LGD estimation. Other models used different rate types (e.g. contractual rate or fixed rate), other proxies (e.g. local cost of equity in the year of default) or featured no discounting of cash flows.

Treatment of multiple defaults

In 45% of the models there was no treatment of multiple defaults. Of the remaining models, only 32% applied a methodology aligned with the EBA Guidelines on PD and LGD estimation (where the time between defaults is counted from the return to normal of the first default).

Additional drawings

Own estimates of conversion factors are required to reflect the possibility of additional drawings after default. In this regard, only 26% of models for which realised LGD was computed for estimation purposes took additional drawings into account both in the numerator and denominator of realised LGD and in the calculation of realised CCF. A further 9% of models included additional drawings both in the numerator and the denominator of realised LGD but not in realised CCF.


Over 60% of LGD models used obligor characteristics (including geography or industry sector). Collateral type was commonly used as a risk driver, both in the assignment to grades or pools and as a sub-component within existing secured components, when the type of collateral determined the haircut value to be applied. Other types of information, such as internal behavioural information or financial information, were less frequently used.

A wide variety of practices were observed in relation to the discounting of cash flows


57


Time-to-workout

A time-to-workout was defined only in around one-third of the models for which an LRA for estimation purposes was calculated. 26% of models calculated it based on analysis of cumulative recovery rates and the remaining 9% defined it using an expert-based approach. Where a time-to-workout had been defined, the most common period used was between four and ten years.

Treatment of incomplete recoveries

Overall, around 30% of models did not have a treatment for incomplete recovery processes. 13% of the corporate models reviewed exhibited a treatment of incompletes that did not include the estimation of future recoveries. Regarding models for the institutions exposure class, it was observed that no institution inferred future cash flows based on observed recoveries.



Overall, in 54% of the models the LRA LGD was not calculated for calibration purposes. This included cases where an LRA LGD was calculated only for the review of estimates. For around 20% of models, the LRA LGD was calculated only at the level of each component, without aggregation to facility level LGD estimates. In general, in 21% of models related to exposures to institutions, a long-run average was calculated at either portfolio or grade level, compared with 29% for corporate models.

Observation period used for the LRA LGD

Regarding the time period used for the calculation of the LRA LGD, the main pattern observed was that 2008-09 was included in all but one model. Differences could, however, be observed per exposure class, as models for corporates tended to consider shorter time frames than models for institutions. This appeared to be motivated by the lower number of internal defaults for institutions portfolios, leading to a greater reliance on data from external providers, the datasets of which often covered the early 1990s as well as more recent years.

Around 30% of models did not have a treatment for incomplete recovery processes.


58

Weighting approach

It was observed that in a large majority of cases (82%), institutions used the default weighted average in the LRA LGD computation, while the remaining institutions used an inappropriate weighting approach.

Downturn adjustment

Approximately 70% of LGD models defined an explicit downturn period. Of these cases, slightly over 80% included the years 2008 or 2009 within this period. In terms of identification of the downturn period, GDP was the most commonly considered macroeconomic factor.

Regarding the types of downturn adjustment, the horizontal analysis provided the following observations:

• in 45% of cases the institution identified a downturn period and used observed values (normally the realised LGD or components of the LGD) in order to derive the downturn adjustment;

• in 18% of cases the downturn adjustment was based on a statistical approach (e.g. impact simulation, confidence interval);

• in 8% of cases the downturn adjustment was based on stressed values for components of the LGD;

• in 29% of cases no downturn adjustment was applied (for example when institutions considered that all the internal data were generated in a downturn period, or cases where institutions considered that there was no impact from an economic downturn).


Of the investigations which included models for defaulted assets in their scope it was observed that in most cases (60%) both ELBE and LGD in-default were being assigned to defaulted exposures. In 13% of cases only ELBE was being calculated and in 5% of cases only an LGD in-default was being calculated. In the remainder (22%), neither an ELBE nor an LGD in-default value existed.

With regard to ELBE, in 29% of cases there was a dedicated model in place, while in 44% of cases ELBE was set as equal to the specific credit risk adjustments for the exposure. Of the cases where a dedicated model was in place, 62% based their expected loss estimation on the LGD performing model. Of the cases with a standalone model, the majority used empirical evidence based on internal data in the ELBE estimation.

With regard to LGD in-default, there was a dedicated model in place in only 20% of cases. A more common approach (44% of cases) was to rely on a pre-existing ELBE

Approximately 70% of LGD models had an explicitly defined downturn period (which typically included the year 2008 or 2009).

In most cases (60%), both ELBE and LGD in-default were being assigned to defaulted exposures


59

and directly estimate the unexpected loss (i.e. the difference between LGD in-default and ELBE) through the calculation of add-ons. Of the models with an LGD in-default calculation, 48% considered a specific downturn component, 38% considered a component for unexpected losses, and 21% included an explicit MoC component.


In 35% of cases the approach used by the institution to account for uncertainty of the estimation was based on conservative assumptions, while in 43% of cases an MoC was applied explicitly at grade or portfolio/calibration level. For 22% of the cases, neither an MoC nor conservative assumptions were considered.

Of the cases where an explicit MoC was considered, in 71% a MoC was applied to account for data and methodological deficiencies, in 14% a MoC accounted for representativeness uncertainty and in 81% of cases it accounted for the general estimation error.


In total, there were around 1,700 findings from the TRIM investigations relating to models for LDPs, based on the 75 assessment reports that were considered for the horizontal analysis.76 This includes additional findings identified as a result of consistency checks and horizontal analyses conducted by the centre of competence on the assessment reports delivered by the inspection teams. These analyses led to additional standalone findings (included in the total number of findings mentioned above) and additional observations, which have been integrated into findings already raised by the inspection teams. In terms of severities of these findings, approximately 14% and 48% were classified as F1 and F2, respectively, while 30% and 8% of the findings were classified as F3 and F4, respectively.

Around 2.4% of the findings included in the supervisory decisions were not related to any binding regulatory requirements; for example, they may have been raised only in reference to draft technical standards or guidelines not already in force.

The average number of findings per investigation was 22, while the maximum was 48. Figure 7 shows the distribution of findings per modelling area and by severity.

76 As noted at the start of this section, 76 investigations formed the scope of the review of LDP models.

However, the outcomes of two investigations at the same institution were combined into one assessment report, so there was a total of 75 assessment reports in the scope of this horizontal analysis.

In total, there were around 1,700 findings raised in relation to models for LDPs


60

Figure 7 Overview of findings for credit risk models related to LDPs



Number of

findings


with F3/F4 findings

Number of

investiga-tions

1,006 74

480 49LGD

PD

24 44 68

21 10 31

10 15 25

36 24 60

68 37 105

53 23 76

64 21 85

89 56 145

31 37 68

50 51 101

40 24 64

39 17 56

36 3 39

156 47 203

116 53 169

F1-F2 F3-F4


Rating assignment processGrade assignment dynamicsCalculation of one-year default rateLong-run average default rate

Calibration methodology



Documentation

Other topics1



RDS completeness



88% 51%

85% 46%

46% 4%

59% 22%

61% 24%

88% 54%

78% 46%

72% 45%

65% 23%

53% 28%

80% 51%

71% 41%

43% 24%

51% 20%

84% 65%

Downturn adjustment


ELBE and LGD-in-default


Documentation

Other topics1

69% 45%

57% 31%

71% 33%

59% 39%

22% 12%

27% 22%

13 22 35

15 15 30

24 17 41

22 28 50

7 7 14

4 17 21


61

Source: ECB supervisory information. 1) This covers findings relating to other horizontal topics (e.g. topics related to internal audit, governance or internal validation). 2) This column provides information on the frequency of occurrence of each topic in the TRIM assessment reports considered in this analysis. One assessment report may include several findings within the same topic.

Findings were raised across all of the in-scope risk parameters (PD, LGD and CCF) as well as for overarching areas. The most common findings for the PD and LGD parameters are summarised below.

4.3.2.1 PD-related findings

Rating assignment process

Findings were raised concerning flaws in the rating assignment process and inconsistent application of the assignment process, for example owing to imprecise or undocumented procedures. In addition, a significant number of findings were raised in relation to the improper use of overrides, including missing or inappropriate monitoring, and to flaws in the treatment of ratings of third parties and the use of human judgement.


A significant number of findings were raised in relation to the selection of risk drivers, including instances where not all relevant risk drivers were assessed or errors were identified in the construction of the risk drivers. In some cases modelling assumptions were not properly justified or there were errors in the implementation of the methodology. Additionally, multiple findings were raised in relation to the homogeneity/heterogeneity of obligors or exposures.

158 38

87 75

Ove

rarc

hing

CC

F

8 8 16

6 8 14

10 16 26

18 13 31

64 10

2 2

11 9 20

102 12

121 13

24 4 28

114 15

15 6 21

22 15 37Realised CCF


RDS completeness

Long-run average CCF

Downturn adjustment



Documentation

Other topics1

Other topics1



Documentation

61% 34%

42% 13%

34% 11%

45% 8%

34% 3%

24% 5%

37% 18%

5% 5%

21% 11%

29% 16%

27% 19%

17% 11%

17% 11%

Many of the findings related to PD concern shortcomings in the rating assignment process and the improper use of overrides


62


Findings were mainly related to the lack of analysis of the grade assignment dynamics, and its implications in terms of risk quantification and parameter estimation.

Calculation of one-year default rate

Findings were raised in relation to the incorrect calculation of one-year default rates, including, for example, incorrect definition of the denominator, unjustified exclusions from the one-year default rate calculation and cases where there was an inappropriate treatment of multiple defaults.


Shortcomings were observed in relation to the calculation of the long-run average default rate (e.g. missing analysis on the use of overlapping/non-overlapping time windows, or unwarranted weighting schemes) and in relation to the appropriateness of the period covering the likely range of variability of default rates.

In relation to the calibration methodology, the most frequent type of finding related to calibration assumptions that were not properly justified and deficiencies in the calibration analyses. In both cases a number of high-severity findings were raised.

Moreover, there was a broad lack of compliance with representativeness requirements, particularly in relation to the use of external data for PD quantification purposes.


A number of findings were raised in relation to the lack of robust processes for identifying deficiencies that should be accounted for in the MoC or the lack of processes to quantify the impact of such deficiencies. More generally, several findings were raised concerning the absence of an appropriate MoC framework.


The most common areas with findings included:

• cases where the framework for the review of estimates did not prescribe any predictive ability/back-testing/homogeneity/heterogeneity analysis, or were missing other relevant analyses, for example in relation to data scarcity;

• cases where a regular cycle for the full review of the rating systems had not been defined or implemented, or cases where all the relevant tests had not been performed within this cycle;

Many findings were raised in relation to the framework for the review of estimates, including cases where relevant analyses were missing or not prescribed


63

• lack of thresholds or targets intended to trigger appropriate remedial actions, and the framework not including processes for remediation plans, timelines and responsibilities in the event of adverse validation results.

Documentation and other topics

Findings were raised when the documentation on a topic was incomplete or missing. Of the other (miscellaneous) topics assessed in these TRIM investigations, the areas with the most shortcomings concerned the robustness of the validation function77 and the range of application of the model not being appropriately defined and/or respected in practice.

4.3.2.2 LGD-related findings


Findings mainly related to:

• deficiencies in the treatment of recovery flows and costs, such as the institution not developing an appropriate methodology for the allocation of recovery flows to individual defaulted exposures, or cases where the realised LGD was not based on economic loss or did not include material discounting effects;

• discount rates not being applied, or an inappropriate rate being used;

• inappropriate treatment of multiple defaults, for example not defining an appropriate period of time between the return of the exposure to non-defaulted status and the subsequent classification as defaulted when determining whether the exposure should be treated as having been constantly defaulted.


Just over half of the investigations identified deficiencies with regard to the risk drivers for LGD models, such as missing or irrelevant risk drivers. Also, there were often shortcomings related to improper justification of modelling assumptions.

Reference dataset completeness

Findings were raised with regard to data exclusions not being adequately justified and cases where the institution did not include in the reference dataset all the information needed to estimate the LGD. 77 This includes issues relating to the independence of the validation function, its effectiveness in identifying

weaknesses in the model, and the completeness of the validation report.

A number of findings were raised in relation to the calculation of realised LGD, including cases where recovery flows were not appropriately allocated


64


In 20% of investigations, findings were raised in relation to the non-consideration of incomplete recovery processes in the LRA calculation. In around 60% of investigations, a finding was raised in relation to the estimation of future recoveries, for example where the institution did not analyse the recovery patterns observed on both closed and incomplete recovery processes. Findings were also raised when institutions did not clearly specify a time-to-workout concept.


Just under half of the investigations identified shortcomings in relation to representativeness, where the calibration sample was not representative of the application portfolio or the institution’s processes did not specify any such representativeness analysis. Also common were issues relating to predictive ability and cases where the LGD estimation was not based on realised LGD.

Downturn LGD

Findings were raised across a number of areas, including weaknesses in the quantification of the downturn impact, the identification of the downturn period and insufficient consideration of macroeconomic and credit indicators. Additionally, in some cases institutions did not characterise an economic downturn adjustment in terms of observed losses.


Most of the findings on this topic concerned the lack of a MoC framework or the absence of robust processes for identifying and quantifying deficiencies which should be accounted for in the MoC.


Issues included institutions not having ELBE or LGD in-default estimates, a lack of justification for assumptions in the estimation, and institutions not having clear documentation on the breakdown of ELBE and LGD in-default or the breakdown of the unexpected loss add-on component, when used.

Framework the for review of estimates

The areas with the most findings were similar to those noted above for PD, namely the framework for the review of estimates not prescribing any predictive

Just under half of the investigations identified shortcomings in relation to the representativeness of the calibration sample


65

ability/back-testing/homogeneity/heterogeneity analyses, or where a regular cycle for full review of the rating systems was not defined or implemented.

Documentation and other topics

In around 40% of cases, documentation was either incomplete or missing, particularly in relation to the calculation of realised LGD and of LRA LGD. Other findings related to the robustness of the validation function and the range of application of the LGD model not being appropriately defined and/or respected in practice.

4.3.2.3 Data quality findings

In the same way as for investigations of models for retail and SME portfolios, data quality reviews were a core part of the on-site investigations of credit risk models for LDPs. These reviews included a qualitative review of the IT infrastructure and data management practices applied to the specific rating systems under investigation, a technical assessment of the quality of PD and LGD historical data used in the rating systems, and a review of the input data used for the purpose of the application of the PD model, focusing on rating computation.

In total there were 321 data quality findings from the TRIM investigations related to models for LDP. The average number of data quality findings per investigation was four, while the maximum number of findings per investigation was 13. The findings correspond to 387 identified shortcomings.78 In terms of severities attributed to these findings, approximately 16% and 46% were classified as F1 and F2, respectively, while 31% and 7% of the findings were classified as F3 and F4, respectively.

Figure 8 shows the distribution of shortcomings per data quality topic and by severity. Similar to the data quality review of models for retail and SME portfolios, the findings for LDP models were either categorised under the institution’s technical implementation of the definition of default or in relation to data maintenance.

78 The distinction between shortcomings and findings is the same as under the data quality review of

models for retail and SME portfolios. See Footnote 68.

There were more than 300 data quality findings from 75 on-site investigations of models for LDPs


66

Figure 8 Overview of data quality shortcomings


Shortcomings around data management and data quality processes were the most widespread issues, being raised in 85% of on-site investigations and representing 101 shortcomings (31% of total shortcomings). This topic also received the greatest proportion (40%) of F3 or F4 findings.79

Within data management and data quality processes, the most prominent shortcomings across the investigations related to the control framework (77% of investigations presented shortcomings in this area). These included the insufficiency or lack of controls on relevant data elements of the model(s) under investigation and the inability of institutions to prove the quality of data used for the model(s) under review. Other shortcomings related to the data quality systems, procedures and processes in place (56% of investigations), including issues related to the coverage of the models in scope of the investigations, the relevant IRB data supporting the model(s) or material elements of the IRB data cycle.

Figure 8 also shows a significant number of issues related to IT infrastructure and the outcomes of TRIM technical tests; issues in these areas affected close to three-quarters of investigations (73% and 72%, respectively).

Within IT infrastructure, the most prominent shortcomings across the investigations related to IT systems and databases, transformation and aggregation processes (61% of investigations presented shortcomings in this area, with 60 shortcomings in total). These shortcomings included weaknesses in the IT infrastructure documentation and issues regarding the soundness, safety and security of the systems/databases.

As part of the data quality review additional information was collected on the set-up of the IT infrastructure supporting the models under review. This covered the configuration of the IT set-up, both in terms of degree of centralisation and integration and in terms of the level of automation of the IT infrastructure and data processes supporting the models. Horizontal analysis of this information indicated that when the IT set-ups were more centralised, integrated and automated, the number of 79 It is worth noting that differences were observed in the shortcomings detected in TRIM investigations for

retail and SME portfolio models and for LDP models. However, a detailed comparison between these two parts of the TRIM data quality review is not presented here owing to the differences in the scopes of the two exercises.

Percentage of investi-

gationswith short-comings

Distribution of shortcomings by severityTopic

Number of short-comings

Percentage of investi-

gationswith F3/F4

short-comings

Number of

investiga-tions

98

75

Dat

a m

aint

enan

ce

43 72

19 26

F1-F2 F3-F4

Regulatory compliance

Technical implementation

IT infrastructure – docu-mentation and testing Data management and data quality processesOutcomes of TRIM mandatory technical tests

27% 9%

61% 28%

73% 39%

85% 40%

72% 37%

Def

initi

on o

f de

faul

t

289

7

29

73 11441

68 10133

38 7436

The most widespread issues related to data management and data quality processes

Almost three-quarters of on-site investigations presented issues in the IT infrastructure supporting the models and the outcomes of the TRIM technical tests


67

shortcomings related to the data quality framework and to the more technical aspects of the review80 was lower.

Finally, regarding the outcomes of TRIM technical tests, the majority of shortcomings related to data deficiencies detected by means of mandatory technical tests on PD and LGD historical data used for the purpose of modelling, rather than as input data used for rating computation.

4.4 Market risk

In total, 31 SIs fell within the scope of the market risk centre of competence (i.e. all SIs with internal market risk models), with one investigation carried out for each institution. Correlation trading models did not fall within the scope of the TRIM investigations.

A majority of the findings regarding market risk concern the general features of the VaR and sVaR modelling approach (for example, risk factor modelling and data quality), and this is also where the most severe findings were raised. The TRIM investigations also highlighted several other areas of inconsistent practices among institutions, including, for example, profit and loss (P&L) definitions for regulatory back-testing purposes. The remediation of these findings and the common understanding provided through the ECB guide is expected to reduce such inconsistent practices, thereby contributing to a reduction in unwarranted (i.e. non-risk-based) RWA variability.


According to Article 363(1) of the CRR, competent authorities are able to grant permission to institutions to calculate their own funds requirements for market risk using the internal models approach instead of, or in combination with, the standardised approaches. Permission can be granted for one or more of the following risk categories: general risk of equity instruments, specific risk of equity instruments, general risk of debt instruments, specific risk of debt instruments, foreign exchange (FX) risk, and commodities risk. In this report, institutions that have been granted IMA approval for some but not all risk categories are referred to as having a “partial use model”.

Within TRIM, the ECB considered all 31 SIs with internal market risk models authorised by the competent authorities for regulatory capital calculation. All of these institutions had been granted IMA approval for general risk of debt instruments, and almost all of them also had approval for other risk categories (see Table 6).

80 Including the technical implementation of the definition of default and compliance with the scope and

specifications of mandatory technical tests.

31 investigations were carried out for internal market risk models

About half of the in-scope institutions have approval to use the internal models approach for all six market risk categories


68

As specified in Article 372 of the CRR, the granting of approval for specific risk of debt instruments implies that institutions have to implement an incremental default and migration risk charge (IRC) model.

Table 6 Approved risk categories for market risk models of significant institutions

Approved risk categories for market risk models

Number of institutions

General risk of debt

instruments

Specific risk of debt

instruments

General risk of equity

instruments

Specific risk of equity

instruments Foreign

exchange risk Commodities

risk

14

4

2

2

2

2

Other combinations 5

Source: ECB supervisory information. Note: The table shows the number of institutions with particular combinations of approved risk categories for market risk models.

Further observations on the modelling landscape are provided in the following sections, relating to either VaR and sVaR models or IRC models.

4.4.1.1 VaR and sVaR modelling landscape

Modelling approach

A majority of institutions, 19 out of 31, used historical simulation to estimate VaR and sVaR. Seven institutions employed a Monte Carlo model, and the remainder used either a parametric or mixed approach.

17 of the institutions that calculated VaR by historical simulation, and five employing a Monte Carlo model, used full revaluation for most of the instruments. Most other institutions used a sensitivities-based pricing approach in their VaR model.

Internal back-testing

All institutions performed back-testing at top-of-the-house level. However, in five institutions there was no back-testing of below-top-level portfolios. Among the remainder, it was common (14 institutions) to test one to ten individual portfolios below the top level, while the rest performed back-testing for a larger number of below-top-level portfolios.

With regard to the depth of the back-testing, a majority of institutions (26 out of 31) back-tested between one and five levels of the portfolio hierarchy below top-of-the-house level.

The most common VaR and sVaR modelling approach observed was historical simulation


69

Pricing functions

For half of the institutions, VaR pricing methods were aligned with pricing methods for the economic P&L (41% of pricing methods were the same; 9% of the methods were aligned but used a different parametrisation).

23 institutions used full revaluation for most instruments in their VaR model. The pricing methods employed showed a strong dependence on the complexity of the respective products. More than half of the VaR pricing methods (52%) were analytical or semi-analytical, mostly used for low to medium-complexity products. A quarter of the pricing methods were sensitivities-based and 6% were Monte Carlo methods used mostly for high-complexity products.

99th percentile definition

The TRIM investigations revealed that eight institutions used a definition of the 99th percentile that led to lower capital requirements than the simplified approach described in the ECB guide (in its February 2017 version). In eight other cases, the definition applied by the institution was more conservative.

Data used for VaR and sVaR calibration

22 institutions used a historical period of approximately, but not less than, 250 days for the VaR calculation, while nine institutions used a period of approximately two years.

In most cases the stressed period for the sVaR spanned the years 2008 and 2009. However, five institutions used a different stressed period for at least one of their legal entities.

Risks not in the model engines

Approximately two-thirds (21) of the institutions had an identification process for RNIME, and for ten of those institutions this led to the application of an RNIME add-on either to the VaR or to the capital requirements directly.

4.4.1.2 IRC modelling landscape

General information, positions and scope

A total of 19 IRC models were reviewed, since two of the 17 assessed institutions with IRC models employed separate IRC models for different entities. Of these 19 models, 14 applied a weekly calculation, with Friday being the most commonly used day. All

Three quarters of institutions used full revaluation for a majority of financial instruments within the scope of their VaR model

Two-thirds of institutions had in place a process to identify risks not in the VaR and sVaR model engines

19 IRC models in use at 17 institutions were reviewed in TRIM


70

IRC models within the scope of TRIM were based on a Monte Carlo simulation method.

13 IRC models used a constant position assumption, with a 12-month horizon for the portfolio migration and default simulation. The other models employed a constant level of risk assumption, in which the P&L of the portfolio is simulated over a period shorter than one year, but more than once in order to cover a total period of 12 months.

Copula assumption

14 institutions with an IRC model used a multivariate Gaussian distribution for modelling asset processes, while the others used either a Student’s t-distribution or a mixed approach (i.e. using different distributions for different components of the model).

Migration matrices and PDs

TRIM identified that 11 out of 19 IRC models assumed PDs lower than one basis point, typically for sovereigns with very good ratings, but sometimes also for corporates.

Four institutions did not assume a specific migration matrix for sovereigns, while the others used between two and eight different matrices.

Recovery rates

A majority of institutions estimated recovery rates from data provided by external rating agencies. Other approaches included RRs estimated using IRB models, or RRs provided by the institution’s front office.

Two-thirds of the institutions with an IRC model assumed constant RRs. The others modelled RRs stochastically, at least for corporate issuers.


There were 31 TRIM investigations related to market risk models, of which 17 included the assessment of the IRC models of institutions that used the IMA for specific risk of debt instruments. In total, these yielded more than 900 findings, including additional findings identified as a result of consistency checks and horizontal analyses performed by the centre of competence conducted on the assessment reports delivered by the inspection teams. The average number of findings per investigation was 29; for institutions without an IRC model the average was 22. The maximum number of findings from any single investigation was just over 50. In terms of severities attributed to these findings, approximately 26% and 48% were classified as

The use of zero or very low PDs for obligors in the IRC model was identified in a majority of institutions

Overall, there were more than 900 findings from the market risk investigations


71

F1 and F2, respectively, while 23% and 3% of the findings were classified as F3 and F4, respectively.

Figure 9 shows the distribution of findings per area and by severity.

Figure 9 Overview of findings (market risk investigations)



Number of

findings


with F3/F4 findings

Number of

investiga-tions

152 31

159 31

Reg

ulat

ory

back

-test

ing

Scop

e of

IMA

13 13 26

5 1 6

17 13 30

5 5

3 1 4

6 2 8

23 8 31

6 4 10

17 5 22

13 1 14

19 4 23

9 2 11

29 3 32

41 6 47

21 3

F1-F2 F3-F4

Partial use and specific riskFX and commodities positions in the RBBExclusions and significant shareOwn debt and defaulted debt

CIUs

Legal entities and aggregationDocumentation, general and otherIndependence of units and staffingDocumentation, general and otherAdequacy and completeness

Frequency

Documentation, general and otherAdequacy and completeness

Frequency


10% 3%

87% 19%

52% 10%

32% 6%

55% 13%

26% 3%

45% 10%

29% 13%

52% 19%

23% 6%

13% 3%

16% 0%

55% 32%

19% 3%

65% 32%

Hypothetical portfolios

Documentation, general and other

Scope

Period and business days

Actual P&L

Hypothetical P&L

58% 13%

29% 0%

35% 3%

68% 10%

94% 35%

61% 16%

14 5 19

10 10

14 1 15

28 3 31

39 13 52

20 5 25

Overshootings

Documentation, general and other

35% 6%

23% 13%

12 3 15

16 5 21

91 31

Valid

atio

n an

d in

tern

al b

ack-

test

ing

41 31

Gov

ern-

ance

17 31

Inte

rnal

aud

it


72

Source: ECB supervisory information. 1) This column shows, for each topic, the percentage of investigations in which findings were included in the TRIM assessment reports considered in this analysis (in total, there were 31 investigations for VaR and sVaR, and 17 for IRC). One assessment report may include several findings within the same topic. 2) This category was used as a fall-back for findings which could not be allocated to a specific topic in the market risk chapter of the ECB guide to internal models.

This findings heat map shows the number and severity of findings per topic and also (in the last two columns) the percentage of investigations in which findings were made and the percentage of investigations yielding a finding with a severity of at least F3.

The number of findings varied significantly across investigations and scopes of the investigation (e.g. whether an IRC model was included in the assessment). Most findings were allocated to the following broad topics:

• VaR and sVaR methodology (240 findings);

• regulatory back-testing (159 findings);

• scope of IMA (152 findings);

• IRC methodology (120 findings).

The topics with the fewest findings are those that were not in the main focus of the TRIM investigations for market risk, such as internal audit (17 findings) and internal governance (41 findings).

Below are summaries of the detected deficiencies in selected topics that generated a high number of findings (not all of the topics shown in Figure 9 are included).

240 31

Met

hodo

logy

VaR

/sVa

R

4 2 6

11 5 16

38 11 49

17 14 31

28 14 42

26 10 36

8 3 11

37 24 61

24 14 38

20 11 31

36 5 41

32 14 46

18 5 23Data period and quantile

Data quality

Risk factors

Proxies and regressions

Pricing

Documentation, general and otherPortfolio composition, netting and groupsDistributions and correlation

Ratings, PDs and RRs

Documentation, general and otherIdentification, quantifi-cation, and managementDocumentation, general and other

Documentation, other2

Standardised approach

48% 13%

81% 26%

58% 13%

65% 26%

68% 32%

74% 32%

59% 18%

94% 41%

88% 59%

71% 47%

81% 19%

58% 23%

35% 13%

10% 6%22 31

Oth

er

120 17

Met

hodo

logy

IRC

73 31

RN

IME

16 8 24


73

Scope of IMA

FX and commodities in the regulatory banking book

Most of the findings on this topic concern the methodology applied to derive the FX positions of regulatory banking book (RBB) items, which was either not calculated correctly or not documented adequately. Another issue giving rise to a large number of findings was instances in which market risk FX positions in the RBB were not taken into account in the IMA or were not capitalised at all. Other findings concerned the inadequate internal definition and documentation of the regulatory trading book/banking book boundary and a lack of monitoring of trading intent or other trading characteristics of positions.

Exclusions and significant share

Almost all of the findings on this topic related to the unsubstantiated exclusion from the IMA of individual positions, books or portfolios, which should in general have been covered. Some of these findings related to the incorrect treatment of FX positions or the exclusion of back-to-back transactions despite residual risks arising from those transactions.

Collective investment undertakings

The treatment of CIUs in the IMA was often found to be inadequate. A wide range of different shortcomings were found relating to the need for a clear and well-documented process, the determination of a daily liquid price for the CIUs, and incorrect look-through on the underlying positions of the CIU.

Internal validation and internal back-testing

Adequacy and completeness

Findings on this topic related to shortcomings in the tests and assessments performed as part of the internal validation.


Most findings in this area addressed deficiencies in the internal back-testing programme of the institution. In many cases, the methodology applied to internal back-testing was considered inadequate, or the assessment did not cover all relevant portfolios (i.e. the levels of back-testing were not sufficient).

The treatment of FX and commodities risk from positions in the RRB generated a high number of findings


74

Hypothetical portfolios

Almost all of the findings in this area addressed cases where institutions did not carry out – or carried out only to a very limited extent – the required back-testing on hypothetical portfolios.

Regulatory back-testing

Period and business days

A majority of findings concerned the lack of a specific definition of business and non-business days. This was often linked to institutions performing trading on local holidays without proper risk monitoring on those days, and without taking those days into account in the P&L and/or the VaR. Another group of findings addressed missing P&L figures for specific dates as well as cases where regulatory back-testing, for the purpose of determining the addend to the multiplication factor, was not performed on exactly 250 business days.

Actual P&L

Findings on this topic fall into two main areas. The first concerned the definition and treatment of fees, commissions and net interest income, which must be excluded from the actual P&L. Many institutions had no clear definition of these components and/or did not exclude them from the actual P&L. The second was that many institutions had deficiencies in the treatment of fair value or other adjustments, which were either not documented, not determined correctly, or were not correctly reflected in the actual P&L.

Other findings concerned the incorrect treatment of CVAs and debit valuation adjustments (DVAs) in the actual P&L, inconsistent treatment of passage of time (theta) effect and other shortcomings regarding the calculation of the daily figures.

Hypothetical P&L

There were a wide variety of shortcomings in this area. One area with a slightly higher number of findings was the insufficient alignment of pricing functions, market data and parametrisation between the economic P&L and the hypothetical P&L, as well as the inconsistent treatment of the theta effect in the hypothetical P&L and the VaR. Other shortcomings concerned the alignment of the risk factors taken into account in the hypothetical P&L and the risk categories for which the IMA was approved, in particular for partial use institutions. Some of these institutions included market data pertaining to risk categories for which IMA was not approved.

A majority of findings on regulatory back-testing were related to the calculation of the actual or hypothetical P&L


75

VaR and sVaR methodology

Data quality

A majority of findings for this topic concerned shortcomings in data cleansing processes. Shortcomings in data processing included outlier correction, data filtering and filling approaches that were methodologically not fully correct, insufficiently documented or performed inadequately. Another frequent finding concerned insufficient data quality assessment or validation in the VaR model.

Risk factors

A majority of findings for this topic concerned cases where risk factors were missing or inadequately modelled (e.g. modelled with overly reduced granularity). Other findings addressed insufficient justification or insufficient further assessment of specific assumptions with respect to risk factor modelling (such as the employed return assumptions, or the update frequency of risk factor data). In addition, a substantial portion of findings related to specific model details.

Pricing

Findings on this topic largely related to inadequate pricing methods for particular products in the VaR model (e.g. solely using sensitivity-based approximations), resulting in a failure to meet the requirement for the internal model to capture accurately all material price risks. Other findings concerned cases where validation activities regarding the adequacy of pricing methods in the VaR model were insufficient or missing.

IRC methodology

Ratings, PDs and recovery rates

Most of the findings in this area concerned unjustified or inaccurate RR or PD values. One group of findings concerned PDs close to or equal to zero (i.e. lower than 1 basis point) without proper justification, typically occurring for sovereign obligors in the highest quality rating classes. Other findings addressed an inconsistent assignment process for PDs or RRs or cases where some of these values were set manually without properly documented processes.

A majority of market risk findings were related to VaR and sVaR methodology


76

Distributions and correlations

These findings addressed a broad range of shortcomings with respect to the specific modelling choice of the institution. One area with a particularly high number of findings was cases where there was insufficient or no justification of crucial modelling approaches such as the copula assumptions, of the choice of risk factors or of the correlation assumptions themselves, which must be supported by analysis of objective data in a conceptually sound framework. Other findings mainly concerned the data set for calibrating correlations, which was often found to be of bad quality and/or not sufficiently validated.

Risks not in the model engines81

Identification, quantification and management

Most findings on this topic concerned cases where the quantification of RNIME was missing or inadequate. In many such cases, not all risks were quantified, or the quantification did not consider the cumulative impact of RNIME. Less frequently, institutions were found not to have an RNIME framework in place at all (or at least not for their IRC model), or were found to have a framework that did not cover the complete set of potential RNIME. Some institutions did not set up an appropriate threshold system for determining substantial single RNIME or large cumulative RNIME or for triggering further action.

4.5 Counterparty credit risk

At the beginning of the TRIM project, eight SIs had approval to use the internal model method (IMM)82 to calculate CCR exposure. Hence eight on-site investigations were carried out under TRIM – one investigation for each of the institutions within the scope of the CCR review.

The methodological topics covered in the horizontal analysis presented in this section follow the structure used in the ECB guide, including trade coverage, the margin period of risk (MPOR)83 and cash flows84, collateral modelling, modelling of initial margin, maturity, number of time steps and scenarios, calibration frequency and stress calibration, and validation of the models. 81 The ECB clarified in TRIM that based on the provisions of the CRR, it considers that risks not captured in

the model engines are a component of the IMA for market risks and should be addressed in a dedicated RNIME framework. Diverse practices were observed in relation to the treatment of RNIME, and the ECB therefore clarified its understanding of this topic in the ECB guide.

82 The IMM is an approach available to credit institutions under which they may use internal models approved by banking supervisors in the calculation of their own funds requirements for counterparty credit risk.

83 MPOR is defined in Article 272(9) of the CRR as the “time period from the most recent exchange of collateral covering a netting set of transactions with a defaulting counterparty until the transactions are closed out and the resulting market risk is re-hedged”.

84 For example, coupon payments and transaction maturity payments.

Eight model investigations were performed for CCR


77


This section provides an overview of the CCR modelling and validation landscape for the eight SIs with IMM approval that were within the scope of TRIM review. The observations on the modelling landscape are categorised in accordance with the main topics of assessment under TRIM, which are also aligned to the contents of the ECB guide.

Scope of the IMM

Scope of approval

The scope of approval of the IMM can cover a range of transaction types and asset classes. For derivatives, the scope of approval covered, in most cases, all asset classes, with three exceptions relating to inflation (for two institutions) and commodities (for one institution). Similarly, for securities financing transactions (SFTs), most institutions had an IMM that mainly covered bond and equity underlyings. Some institutions excluded (“carved out”) transactions from the IMM method that cannot be fully revalued in the IMM or that exhibit large deviations in IMM market value at t0 compared to benchmark values.

Margining

Margin period of risk

The position of the MPOR relative to the exposure date along the chosen set of time grid points can be modelled using either a backward or forward-looking approach. Of the eight institutions examined, six employed backward-looking MPOR modelling, while the others used a forward-looking approach.

With regard to the length of the MPOR, in two cases institutions used a length equal to the regulatory floor, while in four cases the length was at least equal to the floor.

The treatment of trade-related cash flows (TRCFs) that are paid during the MPOR where no margin call can compensate for the exposure increase (“collateral spikes”) was very varied; some institutions ignored such payments, while others included them in the effective expected positive exposure (EEPE) either partially or on a case-by-case basis, depending on, among other things, the time grid used.

Variation margin

All eight institutions accounted for the variation margin directly when calculating expected exposures. However, they made different assumptions regarding the future composition of the collateral. Four institutions assumed the current collateral composition would remain, while two institutions assumed that future collateral would

For derivatives, the scope of the IMM covers, in most cases, all asset classes

A majority of in-scope institutions use a backward-looking approach to MPOR modelling


78

consist only of cash in the simulation currency, and the rest used different assumptions.

Initial margin

With regard to the initial margin, there was limited use of a dynamic initial margin model within the IMM. Half of the institutions used a “flat” approach (depending essentially on whether cleared or un-cleared instruments were being considered).

Maturity

General framework

Article 162 of the CRR establishes the methods for calculating the maturity (M) parameter of the risk weight formula under the IRB approach, where M is mapped to one EAD, i.e. one netting set. Institutions usually applied paragraph (2)(g) of Article 162 of the CRR in relation to derivatives, thereby establishing an effective floor for M of one year. For SFTs where the longest contractual maturity of the netting set was less than one year, paragraph (2)(d) of Article 162 of the CRR was usually applied. However, in both cases a minority of institutions followed a different approach allowed under Article 162 of the CRR. Where allowed, all of the in-scope institutions applied a one-year maturity cap in accordance with paragraph (2)(h) of Article 162 of the CRR.

Time grid points and scenarios

Time grid points

As the CRR does not specify the exact number of grid points (defined as the time steps for which expected exposure is calculated) or their position, the approaches used by institutions differed significantly. Among the eight in-scope institutions, a majority had implemented a static set of grid points only, while the remainder used a mixture of static and dynamic grid points. Across the institutions using a static approach, the number of grid points varied considerably, with three institutions using less than 100 grid points, and two using more than 300 grid points.

Number of scenarios

Similarly, the total number of scenarios used to estimate the expected exposure time profiles varied significantly. Four of the assessed institutions used between 1,000 and 2,000 scenarios, while the remainder used between 3,000 and 5,000 scenarios.

The approach used to define the number of grid points and their position differs significantly across institutions


79

Calibration

Calibration frequency

Differences were seen in the calibration frequency for volatility and correlation parameters for the risk factors simulation processes, with half of the institutions having monthly or more frequent calibration and the remainder being less frequent.

Stress calibration

When calibrating stress periods, a majority (five out of eight institutions) used just one period (the one at group level), and two institutions used multiple periods (group and solo). Regarding the identification of the stress calibration period, there was also an almost equal three-way split between those institutions that identified first the three-year stress period for the IMM, those that identified first the most severe one-year stress period for the advanced CVA (A-CVA) approach, and those that employed a mixture of both approaches.

Validation

Validation approaches

Many institutions are currently implementing a comprehensive independent validation framework across risk types. However, owing to the technical specificities, both back-testing and the benchmarking of IMM pricing functions were still mainly under the responsibility of model development. Internal model validation provided an independent challenge of the underlying methodologies and a review of the analysis of outcomes for most of the in-scope institutions. In four cases, however, validation was still not considered to provide a sufficient challenge to validation tasks like back-testing and benchmarking if they operate under the responsibility of model development.

Back-testing

All of the in-scope IMM institutions applied back-testing at risk factor level, and half of them also applied it to actual and/or hypothetical trades. At portfolio level, four used both actual and hypothetical portfolios in their back-testing framework, while the rest used only one of the two approaches.

The number and share of back-tested risk factors and portfolios varied significantly across institutions. Unsatisfactory results were observed in five institutions, especially for asset classes with a high volume of single name underlyings (e.g. credit and equity).


In total, there were 236 findings from eight TRIM investigations related to CCR models, including additional findings identified as a result of consistency checks and horizontal analyses performed by the centre of competence conducted on the

Progress is being made in implementing effective and independent validation, but this remains a challenge

There were more than 200 findings from the CCR investigations


80

assessment reports delivered by the inspection teams. The average number of findings per investigation was 29, while the maximum number of findings from any single investigation was over 50. In terms of severities attributed to these findings, approximately 27% and 50% were classified as F1 and F2, respectively, while 22% and 1% of the findings were classified as F3 and F4, respectively.

Figure 10 shows the distribution of findings per modelling area and by severity.

Figure 10 Overview of findings (CCR investigations)

Source: ECB supervisory information. Note: Where a finding relates to multiple topics (which applies to 18 findings), the finding is counted under each relevant topic on the chart.

The topics with the highest number of findings were validation (60 findings) and governance (51 findings). The number of findings for trade coverage, MPOR, collateral, initial margin, and risk factors and calibration ranged from 20 to 26.

The most relevant (in terms of severity) and common shortcomings observed per topic are summarised below.

Scope and trade coverage

Findings were related to an insufficient coverage of the IMM with respect to the proportion of transactions or of RWA covered by the IMM. Inadequate exclusion or inclusion of transactions in the scope of the IMM was also observed. Finally, there were some cases of large persistent and non-remediated pricing differences.

MPOR and trade-related cash flows

Some findings concerned the length of the MPOR, which was shorter than the regulatory floors, before or after taking into account the conditions that increase the

Percentage of investigations with findingsDistribution of findings by severityTopic

Percentage of investigations

with F3/F4 findings

7 1 8

36 15 51

45 15 60

20 6 26

5 5

10 2 12

17 3 20

20 5 25

12 10 22

19 6 25

F1-F2 F3-F4

Scope and trade coverage

MPOR and TRCFs

Collateral modelling

Initial margin modelling

Maturity

Time steps and scenarios

Risk factors and calibration

Validation

Governance

Other

100% 38%

88% 63%

88% 38%

63% 38%

88% 25%

38% -

75% 38%

100% 63%

100% 63%

63% 13%


81

MPOR length. Another relatively frequent issue was the treatment of TRCFs, including inconsistencies in the default management process and consequences of the interpolation/extrapolation methods owing to a coarse time grid.

Collateral modelling

Findings were mainly related to overestimation of actual collateral value, inappropriate accounting for the collateral composition, and divergences between actual and modelled collateral.

Initial margin modelling

Shortcomings were mostly due to divergences between actual and modelled initial margin, and insufficient accounting for the contractual terms.

Maturity

Findings were related to the M parameter calculation, mainly for SFTs, or using the wrong M formula for IMM exposures.

Time steps and scenarios

Findings on this topic were mainly related to the impact of the granularity of the time grid and of the number of scenarios on the accuracy of the exposure calculation.

Risk factor modelling and calibration

Findings were related to weaknesses in the assumptions of the stochastic processes used and the calibration of their parameters (in particular the volatilities) and to the length of the stress period and the corresponding stress calibration.

Validation

Weaknesses in the scope and depth of the validation tasks were observed. Some weaknesses were related to back-testing owing to inappropriate coverage, missing levels or risk measures (e.g. exposure) or a lack of follow-up action.

Validation and governance were the topics that generated the highest number of findings


82

Governance

Some severe cases of inadequate or missing documentation and processes were observed. There were also findings related to insufficient staffing of various units and unclear responsibilities.

Targeted Review of Internal Models – Overview of supervisory follow-up

83

5 Overview of supervisory follow-up

5.1 Approach to evaluating the supervisory follow-up

Section 4 describes and categorises the findings raised during TRIM on-site investigations. The TRIM project reduces unwarranted (i.e. non-risk-based) RWA variability by requiring institutions to return to compliance in the case of findings related to binding regulation, and by recommending that institutions change their approach in the case of non-alignment with non-binding or not yet binding requirements. Consequently, the supervisory decisions issued as a follow-up to TRIM investigations contain limitations, obligations or recommendations depending on the basis for the finding.

Obligations can be further categorised as follows:

1. to make a change;

2. to make a change, unless further evidence can be provided which clarifies that no further action is required;

3. to provide further analysis or justification, or to better document or report on an aspect of the institution’s internal models.

The first category of obligation can have a direct impact on the RWA calculation and on unwarranted RWA variability if changes directly linked to risk parameters are required. It can have an indirect impact if the changes affect the governance and frameworks supporting the use of internal models. However, all three categories of obligation are needed to ensure the sustainable use and sound implementation of internal models in line with regulatory requirements.

Obligations have, in principle, to be fulfilled within a certain deadline. However, in some cases a finding might warrant immediate action, for example in the event of a clear underestimation of RWA. In this case, a limitation is used in addition to an obligation. Limitations restrict or modify the use of a model. A restriction can, for example, prohibit the use of the model for certain portfolios, whereas a modification can, for example, prescribe changes to the values of certain model parameters or to the calculated own funds requirements.

An obligation is therefore the primary means of setting out how a finding needs to be addressed, while a limitation is usually linked to an obligation and should compensate for the effect of a deficiency (i.e. the underestimation of own funds requirements resulting from it). The obligation remains in force until full compliance therewith, as the specific case of non-compliance with regulatory requirement must be remediated, whereas the effect of the limitation only applies for the period up until the corresponding obligations are fulfilled. Therefore, when describing how the supervisory follow-up of TRIM has contributed to reducing unwarranted RWA variability for credit risk (Sections 5.3 and 5.4), the main focus will be on obligations

Different types of obligations are needed for different findings

The focus of this section is on how obligations are used to address findings


84

and recommendations as the tools through which TRIM will have a permanent impact on the calculation of own funds requirements. Conversely, in the analysis of the impact of the TRIM supervisory decisions on bank capital (Section 5.5), the discussion will focus on limitations to highlight both the quantity of material issues discovered through TRIM investigations and the immediate actions taken to address underestimations of capital requirements.

For further explanations of the approach, see some selected examples of the supervisory follow-up for PD and LGD findings in Section 5.6.1.

5.2 Supervisory follow-up for general topics

As explained in Section 4.1, for general topics, non-model-specific requirements were investigated via 55 on-site visits. The 639 deviations addressed with recommendations are not included in the chart below, which only shows the 45 findings requiring obligations.

Chart 1 General topics – supervisory follow-up by type

(as a percentage of all obligations)

Source: ECB.

Most obligations are to introduce changes with an indirect impact on risk parameters, such as correcting specific issues regarding roll-out plans, the independence and effectiveness of the validation function, or the management of model changes. However, some “analyse, justify, document” obligations were also used, when the back-testing had not been sufficiently documented, for example. As expected for general topics, measures with a direct impact on risk parameters are rare.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of obligations

Change with indirect impact on parametersChange with direct impact on parameters

Change or explainAnalyse, justify, document


85

5.3 Credit risk –models for retail and SME portfolios

For the purpose of this section, the results from 83 IMIs were analysed.85 The obligations cover the findings already described in Section 4.2; the focus is on the PD and LGD risk parameters.

5.3.1 Supervisory follow-up for PD: obligations and recommendations

An overview of the supervisory follow-up according to a high-level categorisation of observed findings for PD models is provided below. While there often is a one-to-one relationship between obligations and findings, it is also possible that obligations cover more than one finding or one finding is covered by several obligations. The analysis below counts obligations several times (and thus contains duplicates) in cases where findings relating to different topics are linked to the same obligation. The analysis also includes recommendations in the event that no obligations were raised.

Chart 2 PD findings – supervisory follow-up by type

(left-hand scale: percentage shares; right-hand scale: number of obligations and recommendations)

Source: ECB. Note: This breakdown takes into account that some obligations/recommendations are related to multiple topics; in this case such obligations are counted in each category.

These categories are ordered according to frequency. The categories are derived from the finding underlying the obligation or recommendation and, thus, the same categories have been used as in Section 4.2.2.1, with the exception of rating grade assignment, which was not analysed further owing to the low number of obligations. Roughly speaking, the number of obligations and recommendations are similar for the last four categories, but higher for the risk differentiation category.

Across these categories, it can be observed that, in order to ensure compliance with regulatory requirements, obligations of the types “change or explain” or “analyse, 85 In Section 4.2, 85 investigations were mentioned. However, for two of them, no decisions were issued.

0.0

0.5

1.0

1.5

2.0

2.5

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Risk differentiation Margin of conservatism Long-run averagedefault rate

Calculation of one-yeardefault rate

Framework for reviewof estimates

ChangeChange or explainAnalyse, justify, documentRecommendationNumber of obligations and recommendations per investigation (right-hand scale)


86

justify, document” are more common. In particular, for risk differentiation, institutions have to justify their selection of risk drivers and their modelling assumptions, and for the calculation of the long-run average default rate, institutions have to explain why the historical period that has been selected is representative of the expected variability of default rates. Consequently, “explain and change” and “analyse, justify, document” obligations are imposed more frequently for those categories. Those obligations are less frequent in other areas such as the calculation of the one-year default rate, which is less reliant on qualitative considerations. In addition, for the MoC and review of estimates categories, the high proportion of “change” obligations highlighted the need for institutions to do more work to meet the regulatory requirements at the time of the TRIM investigations.

Where findings were only linked to guidance provided by the competent authority (according to the ECB’s understanding thereof, as outlined in the ECB guide to internal models), no obligations were imposed. In such cases, only recommendations were made. However, this pattern is more relevant for LGD than PD and the number of cases featuring only recommendations is low for PD.

The regulatory requirements for internal models do not only specify how internal models should be used to derive risk parameters, they also establish which regular processes are required to ensure the ongoing compliance of models. Among the “change” obligations imposed for PD topics, there are obligations that have a direct impact on PD estimates, such as to re-estimate the model or to add a specific MoC. Other obligations, such as to make improvements to validation activities, would have an indirect impact. Chart 3 presents a summary of whether “change” obligations affect risk parameters directly or indirectly.

Chart 3 PD findings – Impact of “change” obligations on risk parameters

(percentages)

Source: ECB. Note: This breakdown takes into account that some obligations/recommendations relate to multiple topics; in this case such obligations are counted in each category.

For the first three categories, obligations with a direct impact on PD estimates are used in the majority of cases requiring a “change” obligation. For MoCs, different scenarios are possible. When a specific MoC was missing (often the MoC to account

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Risk differentiation Long-run averagedefault rate

Calculation of one-yeardefault rate

Margin of conservatism Framework for review ofestimates

Obligations with a direct impact on parametersObligations with an indirect impact on parameters


87

for the general estimation error), institutions were asked to produce an MoC which would have a direct impact on the risk parameter. However, when an institution did not have a framework for MoCs, the institution was asked to produce such a framework, which would have only an indirect impact on the risk parameter. Finally, a missing or incomplete framework for the review of estimates would lead to an obligation to remediate this non-compliance, which would have an indirect impact on risk parameters, as it would improve ongoing model maintenance.

5.3.2 Supervisory follow-up for LGD: obligations and recommendations

Chart 4 provides an overview of the supervisory follow-up for LGD according to a high-level categorisation of the findings raised.

Chart 4 LGD findings – supervisory follow-up by type



Chart 4 illustrates the different topical categories of supervisory measures for LGD, which are organised according to the average number of obligations/recommendations per investigation. It also shows the type of follow-up measure applied for each category, e.g. a recommendation or a specific type of obligation.

Most topics are generally equally represented, with between 0.5 and 1 obligations/recommendations per investigation on average. However, more than three obligations/recommendations per investigation were imposed on average concerning the calculation of realised LGD.

Similar to the results for PD, in cases where institutions have to make justified modelling choices, there is a higher number of “analyse, justify, document” or “change or explain” obligations. This is notably the case for risk differentiation, for which

0

1

2

3

4

0%

20%

40%

60%

80%

100%

Cal

cula

tion

of re

alis

edLG

D

Ris

k di

ffere

ntia

tion

Fram

ewor

k fo

r rev

iew

of

estim

ates

Trea

tmen

t of i

ncom

plet

ere

cove

ry p

roce

sses

ELBE

and

LG

D in

-def

ault

Dow

ntur

n ad

just

men

t

MoC

RD

S co

mpl

eten

ess

Long

-run

aver

age

LGD



88

institutions were often required to justify modelling assumptions. Another example is the downturn adjustment, where an obligation may require an institution to justify the selection of the economic downturn period (see Example 2).

Around two-thirds of findings receive a “change” -obligation. For the framework for the review of estimates and, to a lesser degree, the calculation of an MoC, a high share of obligations do not require a change to the model parameters directly, but instead require a change to the framework (Chart 5).

Chart 5 LGD findings – impact of “change” obligations on risk parameters

(percentages)


Chart 5 shows that for the other categories, the share of “change” obligations targeting the model parameters directly is high. An example of this is the calculation of ELBE and LGD in-default, where an absence of a specific model led to an obligation to develop such model, which directly influenced the RWA calculation.86 Another example is the calculation of realised LGD, where the institutions were required to recalculate their estimates, for example if they had not used all information that, in accordance with the regulation, should have been included in the calculation.

The measures related to LGD also include a number of recommendations. This concerns, for example, the discount rate for the purpose of calculating the realised LGD (see Example 1 in Section 5.6.1.2).

86 These findings also often led to a limitation; see Section 5.4.

0%

20%

40%

60%

80%

100%

Cal

cula

tion

of re

alis

edLG

D

Ris

k di

ffere

ntia

tion

Fram

ewor

k fo

r rev

iew

of

estim

ates

Trea

tmen

t of i

ncom

plet

ere

cove

ry p

roce

sses

ELBE

and

LG

D in

-def

ault

Dow

ntur

n ad

just

men

t

MoC

RD

S co

mpl

eten

ess

Long

-run

aver

age

LGD



89

5.4 Credit risk – models for low-default portfolios

For the purpose of this section, the results from 74 IMIs were analysed.87 Following the same approach as the previous section, the obligations cover the findings already described in Section 4.3; the focus is on the PD and LGD risk parameters.

5.4.1 Supervisory follow-up for PD: obligations and recommendations

An overview of the supervisory follow-up according to a high-level categorisation of observed findings for PD models is provided below.

Chart 6 PD findings – supervisory follow-up by type



These categories are again ordered according to frequency and are derived from the finding underlying the obligation or recommendation. They are the same categories as in Figure 7 in Section 4.3.2 with the exception of “other topics” and “documentation”, which have been excluded owing to their horizontal nature. A large number of obligations related to the rating assignment process, which can be particularly complex for LDPs. Not all institutions were able to fully justify their approach or provide complete and accurate documentation, and thus received “analyse, justify, document” obligations. However, where specific weaknesses in processes were detected, institutions were asked to change them, which also led to an equally high number of “change” obligations. For risk differentiation, as already observed in Section 5.3.1, in the context of models for retail and SME portfolios, the need for institutions to justify their methods for building the risk differentiation function leads to “analyse, justify, 87 In Section 4.3, 76 investigations were mentioned. However, for two investigations, no decision was

issued. Another two investigations were covered by the same decision and so the analysis is based on 73 decisions.

0.0

0.5

1.0

1.5

2.0

2.5

3.0

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Ratingassignment

process

Riskdifferentiation

Framework forreview ofestimates

Calibrationmethodology

Margin ofconservatism

Long-runaverage

default rate

One-yeardefault rate

Gradeassignmentdynamics



90

document” or “change or explain” obligations when the institution has failed to convincingly do so. On the other hand, where the regulatory requirements were not met (e.g. a lack of risk heterogeneity between grades, or homogeneity within grades), a “change” obligation was used.

For the MoC and the one-year default rate categories there is a high percentage of “change” obligations, as already observed in Section 5.3.1. By contrast, for the grade assignment dynamics category, “analyse, justify, document” obligations were predominantly used, as many institutions were not able to produce the analysis required in line with the ECB understanding of the regulation.

“Change” obligations can affect risk parameters directly or indirectly, as shown in the chart below.

Chart 7 PD findings – impact of “change” obligations on PD risk parameters

(percentages)

Source: ECB. Note: This breakdown takes into account that some obligations/recommendations are related to multiple topics; in this chart such obligations are counted in each category.

The “change” obligations related to the rating assignment process mostly require the institution to address weaknesses related to processes rather than to methodology, so they have an indirect impact on risk parameters. Similarly, improvements to the framework for the review of estimates, related to the periodicity and content of the reviews, mainly have an indirect impact on risk parameters.

Regarding risk differentiation, for deficiencies such as the omission of important risk drivers or inadequate performance, obligations with a direct impact on risk parameters were imposed to tackle those deficiencies. For process-related deficiencies, such as insufficient procedures to assess the performance of the model, obligations with an indirect impact were imposed.

Furthermore, when institutions did not calculate the one-year default rate or long-run average default rate correctly, applied an incorrect calibration methodology or did not include MoCs as required by the regulation, obligations with a direct impact on the PD parameters were mainly imposed.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Ratingassignment

process

Riskdifferentiation

Framework forreview ofestimates

Calibrationmethodology

Margin ofconservatism

Long-runaverage

default rate

One-yeardefault rate

Gradeassignmentdynamics



91

5.4.2 Supervisory follow-up for LGD: obligations and recommendations

Chart 8 provides an overview of the supervisory follow-up according to a high-level categorisation of observed findings for LGD models.

Chart 8 LGD findings – supervisory follow-up by type



Chart 8 illustrates the different topical categories of supervisory measures for LGD, organised according to the average number of obligations/recommendations per investigation. The other topics and documentation categories have been excluded owing to their horizontal nature. It also shows the type of follow-up measure applied for each category, e.g. a recommendation or a specific type of obligation.

“Change” obligations were predominantly used for most categories. However, “analyse, justify, document” obligations were more frequently imposed for risk differentiation. Typical obligations required institutions to justify the use of human judgement, or to analyse the use of additional potential risk drivers.

Chart 9 provides a breakdown of the “change” obligations.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

0%

20%

40%

60%

80%

100%

Cal

cula

tion

of re

alis

edLG

D

Long

-run

aver

age

LGD

Ris

k di

ffere

ntia

tion

Fram

ewor

k fo

r rev

iew

of

estim

ates

ELBE

and

LG

D in

-def

ault

Dow

ntur

n ad

just

men

t

Mar

gin

of c

onse

rvat

ism

Trea

tmen

t of i

ncom

plet

ere

cove

ry p

roce

sses

RD

S co

mpl

eten

ess



92

Chart 9 LGD findings – impact of “change” obligations on risk parameters

(percentages)


For most of the categories (calculation of realised LGD, long-run average LDG, ELBE and LGD in-default, downturn adjustment, MoC, and treatment of incomplete recoveries), the obligations that were imposed generally had a direct impact on the risk parameter. Obligations to improve the framework for the review of estimates and obligations relating to the completeness of the reference dataset mainly had an indirect impact on risk parameters. Obligations to revise or redevelop the risk differentiation methodology would have a direct impact on risk parameters, while obligations relating to the processes supporting the generation of LGDs, such as an obligation to clarify the treatment of overrides, would have an indirect impact.

5.5 Impact of TRIM supervisory decisions

The purpose of this section is to provide information on the realised or expected impact of TRIM on the capital requirements of the institutions included in the exercise. The breadth and depth of TRIM investigations resulted in a substantial number of findings (more than 5,800), of which about one third have an estimated material impact on institutions’ financial situation, level of own funds or own funds requirements, internal governance, risk control or management. Consequently, TRIM supervisory follow-up resulted in a sizeable increase in the Pillar I capital requirements of some of the institutions inspected.

While the question of the expected impact of TRIM on the regulatory capital of the in-scope institutions is legitimate and straightforward, an answer can only be provided with a number of qualifications. In particular, as indicated in the previous sections, obligations (complemented by recommendations) are at the core of the follow-up conducted within TRIM and are imposed through supervisory decisions. In general, the impact of the follow-up measures cannot be estimated a priori, as it ultimately depends on how each institution decides to address each specific obligation in

0%

20%

40%

60%

80%

100%

Cal

cula

tion

of re

alis

edLG

D

Long

-run

aver

age

LGD

Ris

k di

ffere

ntia

tion

Fram

ewor

k fo

r rev

iew

of

estim

ates

ELBE

and

LG

D in

-def

ault

Dow

ntur

n ad

just

men

t

Mar

gin

of c

onse

rvat

ism

Trea

tmen

t of i

ncom

plet

ere

cove

ry p

roce

sses

RD

S co

mpl

eten

ess


Obligations (complemented by recommendations) are at the core of the TRIM follow-up and are imposed through supervisory decisions


93

practice. Even a posteriori, however, it may be difficult to determine the impact of addressing TRIM-related obligations for two reasons in particular.

• The adjustments institutions make to their models take time to implement (most obligations have deadlines of 12 months or more) and changes to the underlying exposures or macroeconomic conditions may occur in the meantime. This could make it challenging to disentangle the impact on capital requirements of the remediating actions from the impact of portfolio changes.

• Institutions are expected to combine tasks related to remediating TRIM findings with ordinary maintenance work on their models, or with the work required to comply with forthcoming regulatory requirements. This is especially relevant for credit risk models, where institutions can realise valuable synergies between the TRIM follow-up and the model changes needed in response to the EBA’s regulatory review of the IRB approach. As a result, it may be hard to distinguish the possible impact on capital of a specific initiative.

In addition, in some cases TRIM had an impact before an investigation had even begun or a decision had been communicated to the institution. A number of institutions started preparing for the exercise in advance by implementing changes or improvements to their models or, in some cases, by withdrawing their request for a material model change as they realised that the new model would not meet the related regulatory requirements. In other cases, institutions decided to request approval to discontinue the use of some internal models and revert to simpler approaches (e.g. standardised approach or foundation IRB).

With these important caveats in mind, at least part of the impact of TRIM investigations on capital requirements can nonetheless be estimated a priori. This relates to two items contained in some (but not all) supervisory decisions.

• Limitations – As indicated in Section 5.1, these are temporary measures linked to specific findings and obligations and are imposed with the aim of ensuring that any significant underestimation of risk, high uncertainty in the risk estimations, or deficiency in risk management are mitigated while the institution is working on addressing the relevant obligations. Since they are imposed by the supervisor, the impact of limitations can be generally estimated a priori.88 However, it should be noted that these estimates are normally based on data available at the start of the on-site investigation and the actual impact of the limitation may therefore differ from the estimates as a result of changes in the underlying exposures. Also, since not all obligations are linked to a limitation and, in any case, limitations are temporary in nature, the long-term impact of TRIM decisions cannot be fully forecasted on the basis of the estimated impact of limitations.

• Material model changes – Institutions can implement changes to their approved Pillar I internal models over the course of their life cycle for several reasons, e.g. to improve their performance or to respond to supervisory findings

88 For the sake of completeness, it should be noted that some supervisory decisions can also remove

pre-existing limitations, in the event that the assessment indicates that the institution has fulfilled the corresponding obligations to the satisfaction of the competent authority, or replace those limitations with new ones, as appropriate, depending on the findings of the new assessment.


94

or regulatory developments. These changes may or may not have an impact on the capital requirements calculated using the affected models and, depending on the specific case, the impact can result in an increase or a decrease in the capital requirements. When TRIM investigations were combined with the assessment of material model changes that require supervisory approval before being implemented (see Section 3.2.3), the estimated impact of those changes was made available by the institutions as part of the documentation they had to submit when requesting the approval. However, the estimates cannot be considered as fully reflective of the actual impact of the model changes as finally implemented, not only owing to the above-mentioned changes in the underlying exposures, but also because supervisory approvals of model changes are normally issued alongside a number of obligations requiring the institutions to remediate shortcomings identified in the assessment of the proposed model change.

Overall, out of the 253 supervisory decisions issued or in the process of being issued for TRIM,89 74% contain at least one limitation, and 30% contain an approval of a material model change. It is estimated that the aggregated impact of such limitations and model changes approved as part of TRIM investigations will lead to a 12% increase in the aggregated RWA covered by the models assessed in the respective TRIM investigations. This corresponds to an overall absolute increase in RWA of about €275 billion as a consequence of TRIM. At the individual institution level, a median increase of 10%, and an average increase of 26% was observed for RWA covered by each TRIM investigation referenced by those decisions. This corresponds to a median impact of -51 basis points, also corresponding to an average impact of -71 basis points on the CET1 ratios of the in-scope institutions. Beyond these average figures, it should be noted that the impact of TRIM decisions on individual institutions was rather heterogeneous depending on the severity of the findings detected in the investigations. Indeed, the choice and calibration of each specific limitation is, as far as possible, tailored to the specific shortcomings identified and draws on the information and data provided in the assessment report.

For credit risk in particular, several types of limitations were used depending on the specificity of the model and the deficiencies identified. By default, limitations are imposed at the level of the risk parameters (i.e. they are applied at the level of each individual exposure). When this was not feasible or appropriate (i.e. deficiencies affecting several risk parameters at the same time), limitations were imposed at the level of RWA (add-on or floor). Finally, other types of limitations were considered depending on the specificities of the situation.

As an example, in several cases limitations in the form of floors for LGD estimates or at the level of RWA amounts were imposed for decisions related to LDP models.90 89 This number includes 194 supervisory decisions relating to the corresponding TRIM investigations, as

well as 59 second supervisory decisions referring to the same investigations (see Section 3.5 for more background on the issuance of second decisions within TRIM). Another five investigations were excluded from the analysis because the institutions withdrew their applications, and one supervisory decision encompassed two investigations. At the time of publication of this report, 55 decisions were still in the process of being issued. Decisions related to supervisory follow-up for general topics are not included in this analysis.

90 See a selected example of the impact of limitations on variability of LGD for LDPs in Section 5.5.1.

It is estimated that the aggregated impact of TRIM limitations and model changes approved with TRIM will lead to a 12% increase in the aggregated RWA covered by the models in scope of TRIM


95

Indeed, a reliable quantification of the impact of the deficiencies affecting the LGD estimates was often not available owing to the characteristics of those portfolios. Therefore, when there was no reliable quantification and there were general doubts about the adequacy of the LGD estimates (i.e. numerous severe findings calling into question the general reliability of the estimation), floors were applied to the LGD estimates at the level of the regulatory values of the foundation IRB (FIRB) approach (Article 161 of the CRR).

When, there were also general doubts about the adequacy of the CCF estimates or when the bank was already using FIRB values for the CCF estimates, floors of the RWA amounts at the level of the RWA amount calculated using the regulatory values of the FIRB approach (Article 161 and 166 of CRR) were applied. In a few cases, there were general doubts about the adequacy of PD and LGD (and in some cases also CCF) estimates so an RWA floor at the level of the standardised approach (SA) was imposed.

It is important to highlight that approval to continue using models with a floor at FIRB or SA levels was considered as less severe for the institution than a revocation of the use of the LGD (or CCF or PD) model, since the former enables institutions to decide whether they want to continue using the advanced IRB approach (and remediate the identified findings) or whether they want to revert to less sophisticated approaches (in accordance with Article 149 of the CRR). In addition, it is worth noting that the imposition of a floor at FIRB level was necessary to compensate for the severe deficiencies of the model. This is not intended to pre-empt any potential impacts of the implementation of the new Basel standards through the amendments to the CRR.91 Moreover, the scope of TRIM decisions is generally not consistent with the scope of exposures covered by these future requirements. Accordingly, the level of the floor was based on the values of the current FIRB approach set out in the CRR, since it was not deemed appropriate to use the new Basel values for just the LGD estimates and not for the other parameters. In addition, it was necessary to maintain a level playing field with current FIRB banks.

5.5.1 The impact of limitations on the variability of LGD – selected example for LGD of LDPs

Limitations have an immediate impact on risk parameters and the RWA level of institutions. As part of the TRIM exercise a ceteris paribus analysis was conducted to assess the impact of the supervisory measures imposed in TRIM on the variability of LGD estimates for LDPs. The analysis was only performed on LDPs, as this was the only type of portfolio for which it was possible to perform a cleaner comparison and reconciliation between the scope of the EBA benchmarking data and the scope of the models investigated in TRIM.

The analysis was based on portfolio data reported by institutions in the context of the 2020 EBA benchmarking exercise for credit risk92 and information from the TRIM 91 See Basel III: Finalising post-crisis reforms, BCBS, December 2017. 92 Reference date 31 December 2019.

https://www.bis.org/bcbs/publ/d424.pdf


96

assessment reports. These data were analysed to assess the proportion of large corporates and financial institutions (total and unsecured) portfolios covered by the inspected LGD models and the respective exposure-weighted average LGD applied to these portfolios.

Combining this set of data with investigation-level information (in particular on supervisory measures and average portfolio LGDs produced by the inspected models), allowed a proxy93 of the exposure-weighted average LGD including the supervisory measures to be derived for the scope of application of the inspected LGD models and for the large corporates and financial institutions (total and unsecured) portfolios, as reported in EBA benchmarking data. The exposure-weighted average LGD was then compared with the adjusted exposure-weighted average LGD after the simulation of the supervisory measure94 for the total and unsecured portfolios.

The scope of this analysis covered all institutions that were subject to a TRIM investigation for large corporates or for financial institutions including the LGD model, respectively.

Chart 10 and Chart 11 use boxplots to visualise the distribution of the average LGD for the significant institutions in the scope of this analysis (referred to as TRIM SIs).

Two boxplots are presented for each portfolio to illustrate the situation before and after simulation of the impact of the supervisory measures (i.e. before and after TRIM).

93 The proxy of the exposure-weighted average LGD including TRIM limitations at portfolio level was

derived by aggregating (i) the adjusted LGD calculated of the inspected LGD models and (ii) the initial LGD reported at the EBA portfolio level (i.e. LGD at portfolio level ex-ante the effects from the limitation). The weights to be used in this aggregation are the proportions of the portfolio that are covered, respectively not covered, by the relevant model.

94 The TRIM limitation could consist of (i) an LGD multiplier/additive add-on, (ii) a floor to LGD values under the FIRB approach, or (iii) a floor to the RWA level obtained under the FIRB approach. Under case (ii) an average LGD of 45% for unsecured exposures and no impact for secured exposures are assumed. The same assumptions were considered under case (iii). In this hypothetical situation, we would observe a reduction in the dispersion of average LGD driven by the widespread use of floors, owing to the nature of these limitations (floors/fixed LGD value). In practice, this is not necessarily observed in the unsecured portfolio-level data, as the final effect on variability depends on the proportion of the portfolio covered by the LGD models reviewed and, consequently, by the limitation. There is also some interplay with the proportion of secured/unsecured exposures at the total portfolio level, which may also lead to different effects when analysing the total portfolio-level data.


97

Chart 10 Variability of LGD for large corporates portfolios

(percentages)

Source: ECB. Note: The boxplots show the median (central line), the upper (Q3) and lower (Q1) quartiles (top and bottom of the box) and the 90th and 10th percentiles (upper and lower whiskers).

Chart 11 Variability of LGD for institutions portfolios

(percentages)

Source: ECB. Note: The boxplots show the median (central line), the upper (Q3) and lower (Q1) quartiles (top and bottom of the box) and the 90th and 10th percentiles (upper and lower whiskers).

The results of the analysis confirm the positive impact of TRIM on the observed variability of average LGD estimates. In particular, the preliminary results point to a considerable reduction in the dispersion of average LGD, especially for the unsecured portfolios. The effect on this portfolio is particularly relevant as it excludes any possible bias stemming from the different collateral structure of supervised banks and thus provides a more objective picture of the short-term effects of TRIM.

5.6 Reduction of non-risk-based RWA variability through supervisory follow-up

This section uses concrete examples to illustrate how TRIM has contributed to a reduction of unwarranted RWA variability and to an increase in the comparability of

25%

30%

35%

40%

45%

50%

TRIM SIs(before TRIM)

TRIM SIs(after TRIM)



Total portfolio Unsecured portfolio

15%

20%

25%

30%

35%

40%

45%

50%





Total portfolio Unsecured portfolio


98

model outcomes. The examples highlight how these objectives have been achieved through (i) the imposition of supervisory measures during the TRIM follow-up (obligations and limitations) which should be followed by institutions – (see Section 5.6.1), and (ii) the harmonisation of supervisory practices adopted in the context of TRIM investigations (see Sections 5.6.2 and 5.6.3).

5.6.1 The use of obligations as a means of reducing non-risk-based RWA variability – selected examples for credit risk

An important source of non-risk-based RWA variability is the divergent interpretation of the regulatory requirements in the internal models used by the institutions, e.g. the way institutions’ internal data are transformed into risk parameters led, in many instances, to cases of non-compliance with regulatory requirements. The ECB has communicated its understanding of the most relevant elements of the regulation in the ECB guide to internal models. Following a supervisory assessment, deviations from this understanding are addressed with legally binding obligations in the supervisory follow-up. In this section some hypothetical examples are presented to explain in concrete terms how practices that deviate from the regulatory requirements can affect risk parameters and thus RWA variability.

5.6.1.1 Selected examples of the supervisory follow-up for PD

Example 1: calculation of the default rate and long-run average default rate

When calculating the default rate, institutions should consider all non-defaulted obligors at a particular reference date. It is then established how many of these obligors defaulted in the subsequent 12-month period. The default rate is calculated by dividing the number of defaulted obligors by the total number of obligors. In some cases, it was observed that institutions included counterparties in the calculation that had no commitments (considering on-balance sheet exposures, off-balance sheet items and unadvised limits) at the reference date. The inclusion of these obligors is not in line with the regulation and in many cases such obligors are less likely to default, meaning that their inclusion leads to an underestimation of the default rates. When this issue was discovered, to reduce unwarranted RWA variability institutions were given a “change” obligation to amend their default rate calculation; a limitation was also considered.

The LRA DR should be calculated from the average of default rates.95 Consider the following hypothetical example of a calculation over a seven-year horizon.

95 According to Article 180(1)(a) of the CRR, for exposures to corporates, institutions, central governments

and central banks and for equity exposures, institutions shall estimate PDs by obligor grade from the LRA of one-year default rates. According to Article 180(2)(a) of the CRR, for retail exposures, institutions shall estimate PDs by obligor/facility grade or pool from LRAs of one-year default rates.


99

Table 7 Example scenario for the calculation of the long-run average default rate

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7

Number of obligors 500 800 1,200 1,500 1,800 2,100 2,400

Defaults 25 30 35 40 45 50 55

Default rate 5.0% 3.8% 2.9% 2.7% 2.5% 2.4% 2.3%

Source: ECB.

The correct calculation would be to take the average of the seven default rates, which yields an LRA DR of 3.1%. However, the non-compliant calculation of dividing the total number of defaults (in this case, 280) by the sum of relevant obligors (10,300) was also encountered in several TRIM investigations. This would yield a default rate of 2.7%, an underestimation of around 11.5% relative to the compliant calculation. This biasing effect can frequently be observed in cases where institutions grow their portfolio during an economic expansion accompanied by lower annual default rates. To reduce unwarranted RWA variability, where this calculation was observed, institutions received a “change” obligation to amend their LRA DR calculation, and a limitation was considered.

Example 2: choice of observation period

In PD modelling, PD estimation from the LRA DR and the predictive ability of PD estimates are closely related.

The following hypothetical example shows the importance of selecting an accurate period to calculate the LRA DR.

Chart 12 Example of default rate observation period

(percentages)

Source: ECB.

In this hypothetical example, the LRA DR is 1.6% if calculated over the period 2005-17, but only 0.8% if calculated over the period 2013-17.

0%

1%

2%

3%

4%

5%

6%

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Annual default rate Average default rate 2005-17Average default rate 2013-17


100

Institutions have to use at least five years of data for PD estimation.96 However, this does not mean that using exactly five years (e.g. the last five years only) is always sufficient. According to paragraph 83 of the EBA Guidelines on PD and LGD estimation, the LRA DR must also be representative of the likely range of the variability of default rates, which means that bad years also have to be appropriately included. In the example above, the exclusion of the bad years 2008-10 may not be appropriate.

When an institution was unable to justify its choice of period considered representative of the likely range of variability of default rates, a finding was raised requiring the institution to justify the choice of calibration period (“change or explain” obligation) or, where the existing practice could not be justified, to amend their approach accordingly (“change” obligation). Where the institution’s choice seemed reasonable but was inadequately justified or documented, the institution received an “analyse, justify, document” obligation to address these gaps.

Furthermore, where internal models predict PDs that are too low relative to the observed default rates, this misalignment would become visible in either the institution’s own internal validation report, or through the specific tools used by the assessment team. In such cases, supervisory measures – “change” obligations and, in the event of a clear capital underestimation, limitations – would be used to require the institution to correct its approach.

Example 3: risk differentiation – justification of modelling assumptions

In the context of risk differentiation, a wide variety of issues require targeted supervisory follow-up.

An example of a less severe finding is where the lack of justification of modelling assumptions is primarily due to a lack of documentation. For example, it could be that the model shows satisfactory levels of discriminatory power, but the rationale for the driver selection or for specific decisions based on expert judgement has not been fully documented. Such a finding would lead to an “analyse, justify, document” obligation requiring the institution to complete the documentation.

An example of a more severe finding would be a technical error in the modelling process with an unknown impact. For example, the rating process may assign obligors to different rating grades without sufficient evidence that there is a difference in risk between such obligors. In such cases, the institution has to either provide evidence of meaningful risk differentiation or re-estimate the model (“change or explain” obligation).

An even more severe finding would be one where the ECB considers the modelling techniques to be inappropriate, for example because key risk drivers are missing. In such cases, a re-estimation of the model would be required (“change” obligation) and if there is evidence of a significant underestimation of RWA, a limitation would be imposed. 96 According to Article 180(1)(h) of the CRR, for exposures to corporates, institutions, central governments

and central banks and for equity exposures, the length of the underlying historical observation period used shall be at least five years. According to Article 180(2)(e) of the CRR, the length of the underlying historical observation period used shall be at least five years.


101

Through TRIM, the ECB was able to identify and compare the presence and scale of such issues across the supervised banks. This, in turn, enabled a harmonised and proportionate response that led to a reduction in unwarranted RWA variability.

5.6.1.2 Selected examples of the supervisory follow-up for LGD

Example 1: impact of different discount rates on realised LGD

If an institution uses an approach to discounting recovery flows after default that differs from the EBA Guidelines on PD and LGD estimation, a recommendation to be mindful of the guidelines would be issued.97 The table below illustrates the unwarranted variability of realised LGD when using different discount rates. The TRIM follow-up and the implementation of the EBA Guidelines by 2022, which further specify the discounting rate to be used, will help to reduce unwarranted RWA variability. In the example below, Scenario 1 is based on the EBA Guidelines.

Table 8 Impact of different discount rates on realised LGD

Time since default in years Event Exposure value Nominal cash in

Discounted Cash Flow

Discounted Cash Flow

Scenario 1 Euribor: 0.5%

Discount rate: 5.5%

Scenario 2 Risk-free rate +

add-on: 2%

T = 0 Date of default 100

T = 1 Recovery flows 0 0 0

T = 2 Recovery flows 20 17.9 19.22



Sum 30.52 33.27

Economic loss

100 − 30.52 100 − 33.27

= 69.5 = 66.7

Realised LGD

69.5/100 66.7/100

= 69.5% = 66.7%

Source: ECB.

Example 2: measures concerning downturn LGD

Institutions are required to use LGD estimates appropriate for an economic downturn, if these are more conservative than the long-run average.98 In TRIM, measures related to non-compliance with the regulatory requirements were mainly imposed in cases where an institution had not applied a downturn adjustment, or where there were issues with quantifying the downturn impact or identifying the relevant downturn conditions.

Chart 13 shows the differences in own estimates of LGD that result from excluding or incorrectly considering the impact of a downturn period. It presents three scenarios: in 97 The guidelines will become binding on 1 January 2022. 98 Article 181(1)(h) of the CRR.


102

Scenario 1 no downturn adjustment is considered; in Scenario 2 a relatively long period is chosen to calculate downturn LGD; and in Scenario 3 a more conservative estimate of the downturn adjustment is considered. Chart 13 illustrates how the different scenarios may lead to differences in LGD estimates. In TRIM, when the adjustments were not in line with the applicable regulatory requirements, obligations were imposed requiring a change to the LGD estimates. This reduced the unwarranted variability in LGD estimates and, consequently, unwarranted variability in RWA.

Chart 13 Example of the impact of excluding or incorrectly considering the impact of downturn conditions on LGD estimates

Source: ECB.

Where the TRIM assessments found a clear link between the non-compliance and shortcomings in the calculation of model parameters, a “change” obligation was imposed requiring institutions to amend the model parameters. In a number of cases, institutions did not sufficiently explain the rationale behind their choices, which led to “analyse, justify, document” obligations or, in more severe cases, to “change or explain” obligations. In addition, regulatory requirements on downturn LGD are still evolving, which also led to recommendations being issued on this topic. The application of the new EBA instruments will enable further reduction of unwarranted RWA variability, as they will clearly specify the supervisory expectations concerning the treatment of a downturn adjustment.

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

30%

35%

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Observed loss rateEconomic indicatorScenario 1: LRA LGD

Scenario 2: Downturn LGDScenario 3: Downturn LGD (conservative estimate)


103

5.6.2 Reducing non-risk-based RWA variability by implementing a consistent approach for calculating RWA across European banking supervision – selected example for market risk

Harmonised methodology for market risk multipliers

A horizontal analysis was performed on the market risk investigations that resulted in a second supervisory decision99 being issued to cover findings and supervisory measures raised in addition to those already communicated in the first decision.

In this second decision, final measures regarding the qualitative increase in the VaR and sVaR multiplication factors referred to in Article 366(2) of the CRR were adopted in order to ensure that appropriate and proportionate values for the multiplication factors were set consistently across institutions based on a harmonised approach. According to this approach, the proposed increases in the multiplication factors are formulated to reflect, for each institution, the assessment of the overall quality of the VaR models. The number and severity of findings that have not yet been remediated are used as inputs, allowing a qualitative set of weaknesses to be turned into a quantifiable process. Considering the universe of 30 institutions in scope of the second decision, on average, an increase in the VaR and sVaR multiplication factor of 0.63 and 0.65, respectively, was imposed (i.e. 0.63/0.65 added to the initial value of 3 defined by the CRR, respectively for VaR and sVaR). In half of the cases there was an increase in the VaR/sVaR multiplication factor, compared to its previous value (of about 0.5 on average). For the remaining half, there was either a decrease (in the VaR multiplication factor for 8 institutions; in the sVaR multiplication factor for 7 institutions) or no change in the multiplication factor (for the remaining institutions).

As all institutions with approval to use the IMA were subject to this harmonised approach,100 it has reduced non-risk-based variability, while also levelling the playing field.

5.6.3 Implementation of supervisory practices as a means to reduce non-risk based variability – examples of a common methodological framework and several layers of quality assurance

Several tools (namely the ECB guide to internal models, standardised data and information requests, and common ITTs) and multiple layers of quality assurance (including consistency checks101, cross-comparisons and horizontal analyses) were

99 31 institutions were in scope of the market risk on-site investigations. During the project one institution

became a less significant institution, leaving 30 institutions that received a second supervisory decision. See Section 3.5 for more details on second supervisory decisions.

100 The proposed increases in the multiplication factors are considered appropriate and proportionate to compensate for the individual model shortcomings and the potential underestimation of own funds requirements consistently across institutions.

101 Consistency checks and cross-consistency checks of the produced assessment reports by internal model experts from different NCAs and the ECB.


104

systematically deployed in TRIM to ensure a consistent assessment approach and consistent and comparable outcomes of TRIM investigations.

There were regular exchanges and reviews among model experts in order to prepare consistent supervisory decisions, issued as a follow-up to the on-site investigations requiring institutions to address cases of non-compliance with the regulatory requirements. As non-compliance with the applicable regulatory requirements has, in many cases, contributed to unwarranted (i.e. non-risk-based) RWA variability, the consistent resolution of cases of non-compliance identified within TRIM will contribute to a further reduction in this variability.

For a more detailed description of the TRIM quality assurance processes and control mechanisms please see Section 3.4.1.

Targeted Review of Internal Models – Project review

105

6 Project review

6.1 Achievements and project outcomes

TRIM represents the largest supervisory project conducted within European banking supervision so far and involved a broad range of internal model experts from different NCAs and the ECB. The project was initiated by the ECB with the following primary objectives:

• harmonising supervisory expectations and practices in relation to the internal models used by SIs;

• ensuring compliance of the internal models in use with regulatory requirements, in particular with a view to reducing inconsistencies and unwarranted (i.e. non-risk-based) variability when institutions use internal models to calculate their RWA.

To fulfil these objectives, the TRIM project has delivered the following:

• an ECB guide to internal models was developed and published, setting out the ECB’s understanding of applicable regulatory requirements for Pillar 1 internal models;

• ITTs were developed to facilitate a consistent methodological approach within the TRIM investigations;

• on-site investigations were carried out targeting a sufficiently large number of credit risk models and all approved market risk and counterparty credit risk models;102

• several layers of quality assurance, including horizontal analyses, consistency checks and cross-consistency checks, were implemented to ensure that findings were raised consistently;

• supervisory decisions were issued following the individual TRIM on-site investigations to ensure a granular and legally binding follow-up of the shortcomings identified during the investigations.

Table 9, below, summarises how these deliverables fulfilled the initial objectives of TRIM. In particular, it is worth noting that the wealth of results delivered through the project and the intense, detailed supervisory follow-up initiated with the involved institutions has already played and will continue to play a key role in promoting a level playing field and high quality standards for internal models used by institutions directly supervised by the ECB.

102 With the exceptions listed in Section 3.2 of this report.


106

Table 9 Fulfilment of primary TRIM objectives

Deliverable

Objective ECB guide to internal models ITTs, on-site investigations, supervisory decisions

Ensure compliance with regulatory requirements, thereby reducing inconsistencies and unwarranted (i.e. non-risk-based) RWA variability

The ECB guide provides institutions with transparency on the ECB’s understanding of regulatory requirements, in order to develop a common understanding across SSM countries of the existing regulatory requirements for internal models. Thus, institutions have guidance for developing internal models that comply with the applicable regulation, which should reduce instances of inconsistency.

ECB supervisory decisions were issued as a result of the on-site investigations in order to oblige institutions to address instances of non-compliance with regulatory requirements. These instances have, in many cases, contributed to unwarranted (i.e. non-risk-based) RWA variability. Their removal will therefore contribute to its further reduction.

Harmonise supervisory practices in relation to the supervision of internal models

The ECB guide summarises the common understanding of the ECB and NCAs within the SSM regarding the implementation of regulatory requirements related to internal models in use at SIs. As such, it forms the basis for and a first key step towards harmonising supervisory practices and will continue to be used by the ECB and NCAs going forward as common guidance in the review and approval of internal model use in institutions.

ITTs have been developed for the TRIM on-site investigations to ensure consistent reviews across assessment teams. The TRIM ITTs can also be used by on-site teams in future IMIs. Consistent use and training/education on the use of the ITTs directly supported the ECB’s objective of more harmonised supervisory practices within the SSM.

In addition, several layers of quality assurance (including horizontal analyses) were implemented in relation to the outcomes of the on-site investigations to ensure that findings were raised in a consistent way.

Source: ECB.

Further information relating to the development of the ECB guide and the execution of the on-site investigations are provided in Sections 6.1.1 and 6.1.2, respectively.

Aside from the primary objectives of reducing unwarranted (i.e. non-risk-based) RWA variability and harmonising supervisory practices, further benefits of the TRIM project have been realised, relating to the furthering of supervisory knowledge of the SSM model landscape, the strengthening of supervisory practices, as detailed below.

More detailed understanding of the SSM model landscape

Through the activities of TRIM, a much deeper knowledge of the features and characteristics of internal models across the SSM has been achieved. Horizontal analyses carried out across each of the risk types have facilitated a comprehensive detailing of the modelling landscape.

Through the gathering of a consistent set of information and data on the modelling practices of the institutions, new areas have been highlighted for future investigation or monitoring as an outcome of the horizontal analyses produced by the centres of competence.

Strengthening of the supervisory practices for internal models

TRIM has led to a number of potential enhancements in the way the ECB supervises internal models, an example being the development of a unified methodological framework for the prioritisation and selection of credit risk models to be reviewed. The methodology developed and applied during the TRIM project by the centre of competence on model map and prioritisation can directly inform future work related to the planning and prioritisation of IMIs, e.g. when developing, together with institutions, a road map for their internal model strategy.


107

Another example is the enhancement of the guidance on the granularity and severity of findings, in particular relating to credit and market risk internal models. This guidance is intended to be used by assessment teams in the review of models (post TRIM) and should ensure further consistency in the assignment of severities and findings across IMIs.

In addition, through the centres of competence there have been a number of developments that enhance the framework for internal model supervision. For market risk, a new supervisory methodology has been developed in the course of TRIM. This harmonised methodology shall ensure that the qualitative increase of the multiplication factor103 for VaR and sVaR is set in a consistent and proportionate manner and after a case-by-case assessment. The new methodology for determining this qualitative increase provides an approach that should be continued to be used also for future IMIs after the TRIM project has been finalised.104

Finally, the consistency ensured in the context of TRIM, both in terms of supervisory practices across assessment teams and in terms of modelling practices, should “spill over” to regular IMIs as well as to internal models that have not been reviewed within TRIM. These developments are expected to support institutions in deciding on their model strategies; in particular, they may lead to simplification in current model landscapes – partially driven by the implementation of upcoming regulatory developments – or to corresponding improvements for some less material or less critical models, with institutions also expected to adhere to the principles outlined in the ECB guide for these models.

6.1.1 Preparation of the ECB guide

The development of the ECB guide was intended to provide institutions with transparency on the ECB’s understanding of regulatory requirements relating to internal models. This common understanding across NCAs and the ECB is a key prerequisite to ensuring a consistent use of supervisory practices and standards within the SSM and, as such, the development of the ECB guide is viewed as a key result of TRIM.

A selection of examples of how the ECB guide facilitated a common understanding of internal model requirements is provided in Box 1 below. They illustrate the manner in which the ECB guide has sought to address specific instances where there are a wide range of institutions’ practices that may lead to unwarranted RWA variability.

103 Multiplication factors are used within market risk internal models to conservatively reflect flaws and

shortcomings of the VaR and sVaR models related to the risk categories covered by the model’s scope of application.

104 See Section 5.6.2 for more details on this approach.


108

Box 1 Examples of areas addressed through the ECB guide (not exhaustive)

1. Credit risk – probability of default

The ECB guide has provided further clarity on the ECB’s supervisory expectations for the estimation of the long-run average PD. This should be estimated from the “long-run average of one-year default rates”. However, a range of practices have been observed, such as the use of only five years of data or a full data history with no consideration of the representativeness of that period. The ECB guide provides clarity on aspects that institutions are expected to consider when assessing the representativeness of the period chosen, as well as specific references to the EBA Guidelines on PD and LGD estimation, so that institutions are aware of the ECB’s understanding of the relevant CRR requirements and the EBA Guidelines.

2. Credit risk – loss given default

The treatment of incomplete recovery processes has historically been subject to different interpretations by institutions (for example, not considering these cases entirely, only considering the recoveries that have been realised so far with no estimation of future recoveries, or by estimating future recoveries). The ECB guide has provided clarity on how institutions are expected to analyse incomplete recovery processes and extract information relevant for LGD estimation, including in specific cases such as where the collateral has been repossessed but not yet sold.

3. Market risk – regulatory back-testing

Regulatory back-testing is mandatory according to the CRR and has a direct impact on the amount of own funds requirements via the back-testing addend. The ECB guide has sought to clarify the ECB’s understanding of the back-testing framework in terms of the scope and the definitions used, the methodology for calculating actual and hypothetical P&Ls, and the counting and analysis of overshootings (cases where the P&Ls are larger than the risk numbers). In the case of overshootings, the ECB guide was able to provide concrete examples of where overshooting notifications could (or could not) be withdrawn, leveraging on the experience and examples collected through TRIM.

The development of the ECB guide required consultation with a range of stakeholders in order to finalise its content. This included public (industry bodies, institutions) and internal alignment as well as collaboration with NCAs. The number of comments received through the public consultation process indicates a high degree of engagement and willingness of institutions and industry bodies to participate in the development of the ECB guide.

Box 2 below illustrates how the quality of the ECB guide was enhanced through the public consultation process.


109

Box 2 Public consultation on the ECB guide

Feedback was gathered on the ECB guide in two phases; first, comments were sought from the industry on the first version of the guide in 2017, and subsequently a formal public consultation process was launched in 2018 on a revised version of the ECB guide.

In total, 1,103 responses were received in the public consultation process, with 287 comments received on the general topics chapter, 414 on credit risk, 218 on market risk and 163 on counterparty credit risk. By topic, the areas with the most feedback were:

• General topics: Internal validation (65 comments), Internal audit (60 comments)

• Credit risk: Loss given default (127 comments), Probability of default (101 comments)

• Market risk: Risks not in the model engine (96 comments), Scope of the IMA (34 comments)

• Counterparty credit risk: Trade coverage (47 comments), Validation (38 comments)

The quality of the ECB guide was also aided by the involvement of the ECB editing and legal teams.

Related to this point, the ECB guide included references to EBA RTS and guidelines, even though some of them were not legally binding at the time. Their inclusion was considered appropriate, as the parts of the EBA RTS and guidelines referred to in the guide were considered to express an appropriate understanding of the CRR. However, if the RTS undergo significant change as part of their adoption by means of a delegated regulation, the ECB guide may also require revision.

Finally, the ECB guide deliberately focused on a selection of areas deemed to require supervisory attention or a common understanding of regulatory requirements and did not attempt to cover the entire universe of internal modelling topics or regulatory requirements. As such, the ECB guide is less explicit in some areas than in others, and it will remain open to further potential revision or amendment over time where deemed appropriate, e.g. as the ECB gathers more information on selected modelling practices, or in the event of further regulatory developments.

6.1.2 Execution of on-site investigations

Under TRIM, the assessment of internal models in use at SIs was based on investigations carried out at the institutions’ premises. Such on-site investigations are typically costly in terms of time, effort and expenditure.105 However, the experience gained from TRIM provides a clear view that this form of investigation should remain the preferred approach to in-depth assessments of internal models, as it has been identified as the most effective approach to detect weaknesses in models in use at an

105 Specifically when compared to forms of assessment that do not require on-site presence, for example

desktop review of model documentation.


110

institution, while more proportionate approaches can be considered, depending on the scope of the investigation and the materiality and criticality of the model at hand.

The performance of the on-site investigations is discussed in more detail below, with a focus on the structure of the investigations, the processes used for horizontal quality assurance, and resources and staffing.

Structure of the investigations

Collaboration was a strong feature of the set-up of the investigations; the ECB’s Internal Models Division, the centres of competence and the TRIM PMO (alongside the ECB’s Planning and Coordination of SEP Division) provided continuous support to assessment teams, before, during and after the on-site activities. JSTs were also involved at key points in the process, to leverage on their knowledge of the institution, while at the same time avoiding an onerous burden on their time.

The consistent structure of the TRIM on-site investigations, along with their centralised planning and design, made it easier to manage and track the logistics and execution of the 200 investigations performed.

Box 3 On-site investigation structure

Following the commencement of on-site activity, each TRIM investigation needed to complete three distinct phases of work in order to conclude the investigation and communicate findings to the institution (prior to initiating the supervisory follow-up and issuing a supervisory decision), namely:

1. On-site phase (from kick-off meeting to delivery of the draft assessment report for consistency checks): this involved the assessment team deploying the ITTs and conducting interviews with selected members of staff of the institution in order to review the model(s) in scope of the investigation as well as drafting the assessment report.

2. Consistency checks phase: these checks allowed verification that the investigation was conducted according to its mandate and the expected assessment methodology, to ensure that similar issues led to similar findings and severities.

3. Report finalisation phase (from the end of the consistency checks until the finalisation of the report, including the cross-check analyses and the assessment of the comments received from the institution): this phase involved a feedback process to share the consistency checked assessment report with the institution for its comments, allowing the assessment report to be finalised, forming the basis for supervisory decisions. Furthermore, an additional layer of consistency to ensure a harmonised approach to the assessment of the analysed topics was implemented across credit risk investigations in the form of cross-check analyses of the assessment reports performed by different members of the centre of competence than the ones responsible for the consistency checks in the first place (cross-consistency check). The results were discussed at the level of the centre of competence, together with the potential disagreements raised during the consistency check phase. In several


111

cases these analyses led to the identification of additional observations and findings, which were included in the decision phase.

Horizontal quality assurance processes

Given the magnitude and complexity of the project, a comprehensive and multi-layered quality assurance framework was put in place to ensure consistent and comparable outcomes, particularly in relation to the performance of on-site investigations. This framework played a significant role in the successful execution of the TRIM investigations and ensured the application of a consistent assessment methodology. The framework included the following aspects:

1. a common, standardised methodological framework that served as the basis for the harmonised execution of TRIM on-site investigations, namely the ECB guide and ITTs;

2. consistency checks and cross-consistency checks of the produced assessment reports by internal model experts from different NCAs and the ECB;

3. horizontal analyses performed after the finalisation of the assessment reports referring to similar investigations. In some cases, the outcomes of these analyses have also been included in supervisory decisions, enriching the set of findings raised by the assessment team and fostering consistency in the outcomes of TRIM for different institutions.

The framework is described in more detail in Section 3.4.1.

Resources and staffing

The TRIM project was delivered with a high degree of support both from external resources (primarily consultants) and from the efforts made by the ECB and some NCAs to increase capacity by hiring of new staff or by redeploying existing staff in response to this resource-intensive project.

Regarding internal staffing within European banking supervision, it is worth pointing out that a number of cross-border investigations106 were carried out during the course of TRIM, which has proved valuable in furthering the goal of harmonised supervisory assessment.

Regarding consultancy support, the use of external resources as an integral part of TRIM teams (for on-site investigations, the TRIM PMO and centres of competence) was considered necessary in order to be able to complete the numerous activities of TRIM in a timely manner, in particular given the need to maintain ongoing internal model supervision in parallel with the project.

To avoid potential conflicts of interest and to protect confidentiality, contracts with consultancy firms that were selected to support TRIM included clauses for managing 106 Investigations where the head of mission and part of the assessment team are from a non-home/host

competent authority.


112

potential conflicts of interest, including requirements for contactors to implement strict segregation of teams (Chinese walls) and cooling-off periods, preventing the firm and specifically the project team members from working on any related assignments immediately after the TRIM project. Furthermore, possible conflicts of interest were managed by the ECB’s Planning and Coordination of SEP Division on an ongoing basis by regularly checking with the service providers for any potentially conflicting engagements before assigning them to a TRIM investigation, assessing the ex-ante requests for permissions that some providers would submit before engaging in any activity that could give rise to perceived conflicts of interest, and following up on any possible issues detected in this area.

6.2 TRIM’s contribution to improving ongoing and future supervisory work on internal models in the SSM

The objectives of TRIM included, inter alia, contributing to the improvement of ongoing and future supervisory work on internal models in the SSM. The work conducted by the TRIM Harmonisation Board in relation to this objective was organised in four different work streams, according to the specific topics to be covered:

1. Strategy and resources for IMIs after TRIM

2. Methodological developments and related governance

3. Processes and quality assurance

4. TRIM reduction of unwarranted (i.e. non-risk-based) RWA variability

The subsections below provide a summary of the main activities performed and conclusions reached in each work stream.

6.2.1 Strategy and resources for IMIs after TRIM

Already before the launch of TRIM, the existing SSM resources for on-site internal models supervision did not suffice to deal with the number of requests for supervisory approval of new models or material model changes and extensions that are implemented by institutions as part of their routine model maintenance. During TRIM this issue was partly mitigated and a higher number of on-site IMIs could be conducted thanks to systematic and extensive use of consultancy support, as well as additional resources temporarily re-allocated from non-model-related on-site inspections to IMIs.

In 2020 and 2021 this issue has been exacerbated by additional IMI demand stemming particularly from the follow-up work institutions are required to undertake to address TRIM findings and from activities related to the EBA regulatory review of the


113

IRB approach.107 In view of this anticipated increase in IMI demand, work on the strategy and resources for IMIs after TRIM was initiated by the Harmonisation Board in 2018 and concluded with the Supervisory Board decision on this topic in 2019.

The strategy approved as a conclusion of this work combines an increase on the supply side (i.e. the resources available in the SSM to conduct on-site IMIs) with a streamlining of the demand for on-site IMIs.

Regarding the latter, the following measures have been considered as possible tools to efficiently manage IMI demand:

(i) in the short term (2020-2021), maximise synergies between regulatory and supervisory initiatives by combining the implementation of the EBA regulatory review of the IRB approach and the follow-up of TRIM;

(ii) using proportionate approaches for IMIs to increase the efficiency of internal models assessments by also exploring new solutions without jeopardising the overall reliability of the assessment;

(iii) applying strict prioritisation of IMIs according to the SSM strategic objectives for internal model supervision;

(iv) streamlining the number and scope of IMIs performed per year and model, reprioritising bank and supervisory resources to free up additional assessment resources;

(v) supporting the simplification of institutions’ modelling landscape, where appropriate, so as to focus bank and supervisory resources in a risk-based manner.

Meanwhile, follow-up activities intended to further detail the proposed strategy have continued. For example, regarding measure (ii) above, a work stream of ECB and NCA internal model experts has been set up to fulfil the mandate given by the Supervisory Board to elaborate on the possible introduction of an IMI taxonomy that is more flexible than the existing one, which relies either on “JST IMIs” (normally conducted off-site) or on-site IMIs only. The enhanced framework, which would also include lighter approaches to on-site IMIs and central model expert desks performing off-site reviews of a more complex nature than JST IMIs for low materiality models, was presented to the Supervisory Board in 2020, alongside the results of a pilot project conducted by selected JSTs in order to develop, together with institutions, a road map for their internal model strategy.

107 The EBA regulatory review of the IRB approach was launched by the EBA in 2016 and consists of a

number of regulatory technical standards and guidelines that provide more detailed guidance and clarity on several aspects of the applicable rules for internal models. Institutions have to comply with the requirements set out in these new standards and guidelines by pre-defined deadlines, starting at the end of 2020 for the specifications related to the definition of default.


114

6.2.2 Methodological developments

One of the main objectives of TRIM was to harmonise supervisory practices for internal model supervision within the SSM. To pursue this goal, the ECB guide to internal models and a common assessment methodology to be applied by all assessment teams in TRIM (the ITTs) were developed in the preparatory phase of TRIM.

As previously indicated, the guide is intended to continue to be the main reference document for assessment teams when conducting future IMIs, as well as for supervised entities to ensure that their models are aligned with the ECB’s understanding of the applicable regulatory requirements. As such, the guide may also be further updated as necessary in the future, for instance to adapt to new regulatory developments or to cover additional topics that are not currently included.

Similarly, future internal model supervision within the SSM will also benefit from the ITTs developed for TRIM, as these provide a thorough assessment methodology that can be redeployed in future IMIs, either in full or in part, as appropriate, depending on the scope and expected intensity of each investigation, as well as fine-tuned and complemented, as needed, to reflect the specificities of the models under review. In fact, the ITTs have been integrated into the operational guidance detailing the IMI methodology.

6.2.3 Processes and quality assurance

The focus of this work area is on procedural and organisational aspects, including, for example, the possibility of transferring some of the tools and processes developed within the TRIM PMO for use in future IMIs. As no dedicated PMO function is available after TRIM to support ongoing IMIs alongside the existing planning and coordination functions of the ECB and the NCAs, it was acknowledged that, overall, the opportunity to transfer TRIM-specific processes to future model supervision was rather limited.

For instance, standardised data requests were recognised by several heads of mission as a key tool for a successful and efficient on-site investigation. Their use in future IMIs is therefore broadly supported insofar as possible, but the degree of standardisation achievable is generally expected to be lower than in a project like TRIM (where common requests could be deployed across investigations thanks to the harmonised approach to the scope and methodology).

6.2.4 TRIM reduction of unwarranted (i.e. non-risk-based) RWA variability

This work stream covers in particular considerations on how to best assess the desired reduction of non-risk-based RWA variability. Section 5 of this report contains the outcome of the work conducted in this area, illustrating how TRIM achieved its stated objective of reducing unwarranted RWA variability, for example by restoring compliance of banks’ internal models with regulatory requirements.

Targeted Review of Internal Models – Main achievements and conclusions

115

7 Main achievements and conclusions

The TRIM project was launched by the ECB at the beginning of 2016 in close cooperation with NCAs. Its main objective was to confirm the appropriateness of Pillar I internal models in the light of the applicable regulatory requirements and that the results of these models are reliable and comparable. This followed concerns raised regarding unwarranted (i.e. non-risk-based) variability of outputs of models used to calculate regulatory capital requirements and criticism from external stakeholders of the complexity of the models and the resulting opaqueness of the modelling approaches.

7.1 Main achievements of the TRIM project

Within its mandate, the TRIM project has fully achieved its main objectives:

1. to reduce non-risk-based RWA variability within the SSM;

2. to support future supervision of internal models within the SSM.

Reducing non-risk-based RWA variability

Non-risk-based variability can arise from various sources e.g. from regulation not being sufficiently specific to rule out such variability, or from institutions not complying with regulatory requirements. Within the project’s mandate, the ECB has provided transparency on its understanding of these regulatory requirements, through the ECB guide to internal models, which was published for consultation and communicated to all stakeholders. The guide not only provides transparency on the ECB’s supervisory understanding of existing regulation concerning the topics under review in the TRIM project, but also fosters consistency in the implementation of regulatory requirements, while supporting a harmonised assessment of internal models.

In this context, a comprehensive and in-depth assessment framework was developed and consistently implemented in TRIM to assess the compliance of internal models with the applicable regulatory requirements. Under this framework, a common methodological approach – based on standardised data requests and ITTs – has been developed and applied across the 200 on-site investigations performed within TRIM.

Numerous deviations from the regulatory requirements were observed, resulting in more than 5,800 findings across all risk types, of which around 30% were of high severity (i.e. F3 or F4). To ensure that the observed deficiencies were remediated swiftly and own funds requirements were not underestimated during the remediation phase, for all TRIM investigations supervisory follow-up measures and actions were imposed through supervisory decisions containing obligations, recommendations and,

The ECB guide seeks to ensure that supervisory requirements are implemented in a harmonised and consistent manner

Binding supervisory measures were imposed requiring the remediation of deficiencies


116

sometimes, limitations. In some cases these had immediate substantial quantitative impacts on RWA amounts.

In fact, following the TRIM investigations, 253 supervisory decisions have been issued or are in the process of being issued. It is estimated that these decisions (combining the impact of limitations included in and of material model changes approved by those TRIM decisions) will lead to a 12% increase in the aggregated RWA covered by the models assessed in the respective TRIM investigations. This corresponds to an overall absolute RWA increase of about €275 billion as a consequence of TRIM and to a median impact of -51 basis points and an average impact of -71 basis points on the CET1 ratios of the in-scope institutions.

Supervisory decisions issued as a follow-up to TRIM investigations are not intended to reduce or discourage the use of internal models, but rather to bring all those models fully into line with regulatory requirements and to compensate for any identified underestimation of risk. These measures ultimately improve the comparability of model outcomes and thus contribute to restoring the credibility of RWA calculations based on internal models.

Efforts to address the TRIM supervisory follow-up are expected to be made in conjunction with preparations for upcoming regulatory developments. For credit risk models, institutions will have to implement the changes in regulation and interpretation criteria associated with the EBA’s regulatory review of the IRB approach. In addition, the implementation of Basel III through the amendments to the CRR will also simplify the current situation, as it will lead to the decommissioning of LGD models for some “hard-to-model” portfolios, thus further reducing non-risk-based RWA variability.

See Section 5.6 for some concrete examples of how TRIM reduced non-risk-based RWA variability.

Supporting future supervision of internal models within the SSM

As the largest supervisory project launched by ECB Banking Supervision so far, TRIM has delivered a range of benefits and results beyond assessing the compliance of internal models with regulatory requirements.

First, the project has enabled supervisors to gain a much deeper, system-wide knowledge of existing modelling practices and related shortcomings, which will help them to define areas for future investigation or monitoring.

Second, the ECB guide to internal models, developed and published during the course of the project, will be further complemented in the future to reflect regulatory developments and any additional interpretative issues that may arise on an ongoing basis. By complementing regulatory initiatives in the field of internal models from a supervisory perspective, the guide contributes to sounder internal model frameworks and assessments within the SSM.

Institutions are expected to implement TRIM supervisory measures together with changes associated with upcoming CRR and EBA developments

TRIM increased transparency about models in use and contributed to the consistent use of supervisory requirements and assessments


117

Furthermore, the sound governance structure of TRIM facilitated enhanced cooperation with NCAs and has helped foster a consistent approach to the supervision of internal models used by SIs under the SSM.

A similar approach will be used for future supervisory assessments of internal models, which will further contribute to ensuring a level playing field.

Although TRIM has concluded, its legacy in terms of methodologies, supervisory practices and in-depth knowledge of the modelling landscape has been well embedded in the two pillars of regular internal model supervision within European banking supervision, namely IMIs and ongoing model monitoring.108 While the approach developed under TRIM ensured the project objectives were achieved in a defined period of time, the principles underlying that approach, as well as its methodological improvements, can and should be applied to all internal models in place at institutions.

Considering the investment needed to comply with the standards imposed in TRIM, institutions should consider defining an internal model strategy to inform future internal model development or simplifications, particularly when considering where to invest time and resources. This strategy should also extend to non-investigated models, as institutions are expected to remediate similar weaknesses in those models or to consider further streamlining their model landscape.

In summary, within its mandate and through the achievements described in this report and highlighted in the previous paragraphs, TRIM has:

• enabled a deeper, system-wide knowledge of existing modelling practices and the typical shortcomings relating to the use of internal models;

• addressed the deviations from regulatory requirements or underestimations of risk parameters through follow-up measures which ensure that internal models used by SIs are fully in line with regulatory requirements and own funds requirements are not underestimated during the remediation phase; once the numerous supervisory findings that were identified in TRIM investigations have been properly remediated there should be no doubt as to the reliability of these models;

• raised the bar for SIs using internal models and paved the way for satisfactory models to be developed in the future and for the optimisation of the model landscape at European banking supervision level;

• contributed to reducing unwarranted variability of RWA and to maintaining a level playing field across banks under the ECB’s direct supervision as regards the use of internal models;

• contributed to restoring overall confidence in internal models.

108 For more details on the supervision of internal models, please refer to the dedicated page on the ECB’s

banking supervision website.

TRIM restored the credibility of internal models within the SSM

https://www.bankingsupervision.europa.eu/banking/tasks/internal_models/html/index.en.html


118

The on-site investigations carried out under TRIM represent an unprecedented time and resource investment by ECB Banking Supervision and involved a wide range of internal model experts from different NCAs and the ECB. The wealth of results delivered through the TRIM project and the intense, detailed supervisory follow-up to the investigations fully justifies this investment, restoring trust in internal models and providing a framework for enhanced and consistent supervisory assessments going forward.

7.2 Main conclusions of the results of TRIM

After more than four years of intense work, the results of the TRIM project presented in the previous sections have led to the following conclusions, which confirm that consistent implementation of internal models within a supervisory area is possible.

1. Thanks to the detailed supervisory follow-up of TRIM, existing internal models can be considered suitable for the calculation of Pillar 1 own funds requirements.

2. Numerous deviations from regulatory requirements have been addressed through legally binding supervisory measures (limitations or obligations). In some cases these have a substantial immediate quantitative impact, which ensures an appropriate level of own funds in the short term and full compliance with all regulatory requirements in the medium term.

(a) Institutions need to make additional efforts to remediate all deficiencies in a timely manner and to the required standard (they are expected to find synergies with ongoing work to implement new regulatory requirements).

(b) In addition to the supervisory follow-up, institutions’ independent validation and audit functions need to strictly follow up on remedial actions.

3. Going forward, institutions need to continue to invest in the maintenance and development of internal models to maintain the high quality of models achieved through TRIM. In particular, in accordance with the supervisory standards applied in TRIM, the ECB expects the independent internal validation function to be able to ensure an ongoing internal challenge of the performance of internal models, and in some cases its oversight role should be further strengthened.

4. These efforts and investments are expected to support institutions in deciding on their model strategies; in particular, they may lead to simplification in current model landscapes – partially driven by the implementation of upcoming regulatory developments – or to corresponding improvements in some less material or less critical models.

5. Institutions’ efforts need to be complemented by continued intrusive supervisory scrutiny, including:

(a) an adequate and proportionate multifaceted approach to ongoing model monitoring;


119

(b) strict assessment of model changes or initial model approvals in line with the supervisory methodology developed in TRIM.

TRIM provided a proof of concept for consistent internal model supervision within one of the largest supervisory areas in the world. Once all of the TRIM findings have been remediated, there will essentially be a consistent use of high quality standards and a level playing field for all institutions directly supervised by the ECB. As a result, institutions will be also better prepared to deal with the consequences of economic shocks, or the potential challenges posed by the entry into force of a more stringent regulatory framework.

Targeted Review of Internal Models – Appendices

120

Appendices

Supplementary information for Section 3.5 (Project deliverables)

During the course of TRIM, 200 on-site investigations were carried out in relation to risk-type specific topics. Like IMIs for the approval of new models or model changes (at the initiative of the credit institution), the TRIM investigations (at the initiative of the ECB) were composed of a number of distinct phases: defining the scope and preparing for the investigation, the on-site inspection itself (including the drafting of the assessment report), off-site consistency checks, the finalisation of the investigation phase with the delivery of the final assessment report to the institution, and the supervisory decision process arising from the findings identified during the on-site phase.

Figure A.1 Overview of on-site investigation process

Source: ECB.

The average duration of TRIM investigations was around 30 weeks,109 of which, on average, around 14 weeks were dedicated to the on-site phase. The original schedule for the on-site execution phase had to be adjusted in some cases, as some investigations required a postponement and/or an extension, primarily to ensure the availability of appropriate resources for the assessment team. Despite this, the overall execution of the 200 TRIM investigations was successfully concluded within the planned time frame, with some internal adjustments but only very limited reductions in scope.

109 This includes the on-site assessment phase, consistency checks and finalisation of the assessment

report.

Launching phase

• Head of mission appointed and assessment team formed

• Notification letter and data request(s) sent to SI

Preparatory phase

• Operational preparationsfor on-site investigation

• Preparatory analysis of data and information received

On-site phase

• Kick-off meeting

• Analysis of documentation and data

• Interviews with SI staff and walk-throughs

• Finalisation of draftreport

Consistencychecksphase

• Consistency checks performed

Report finalisation phase

• Exit meeting (one day)

• Comments provided by the SI

Draft decision phase

• Supervisory decision

• Finalisation of the assessmentreport

Follow-up of TRIM

investigationsPhase

Key activities


121

• Launching phase: the main activities of the launching phase were to appoint the HoM and team that would carry out the investigation, to agree the scope and timelines with the HoM, and to notify the institution involved of the upcoming review, together with the request for data necessary to perform the assessment.

• Preparatory phase: this phase involved the assessment team collecting the documentation deemed necessary to conduct the review (for example, documentation relating to the development, implementation and validation of the model) as well as checking that the standardised data requested in the launching phase were adequate for the purpose of testing.

• On-site phase: this involved the assessment team deploying the ITTs, reading documentation, performing analyses and conducting interviews with selected members of staff of the institution in order to review the model(s) in scope of the investigation. Regular calls were held between the assessment team, TRIM teams (primarily the PMO and centres of competence) and the ECB’s Internal Models Division in order to track progress and manage potential risks. As part of the on-site phase, a draft assessment report (AR) was prepared, summarising the results of the on-site review.

• Consistency checks phase: members of the relevant centres of competence and of the ECB conducted consistency checks to verify that the investigations had been conducted according to their mandate and that the assessment report had been drafted according to the expected assessment methodology, which involved discussions with the HoM. In general, a “cross-consistency check” format was adopted in TRIM, whereby the centre of competence member in charge of checking a given assessment report belonged to a different NCA than the one leading the investigation (or was an ECB staff member).

• Report finalisation phase: this phase involved a feedback process to convey the contents of the assessment report to the institution (through an exit meeting) and to provide the institution with the opportunity to also comment in writing on the correctness of the facts and findings contained in the report. This step helped to ensure the accuracy of the information contained in the report, in order for the HoM to finalise the document for delivery to the supervised institution and to the ECB’s Internal Models Division for the preparation of the supervisory decision.

• Draft decision phase: ECB supervisory decisions110 are a standard outcome of IMIs and their preparation follows well-established ECB processes. As such, the decision phase is not considered part of TRIM, but pertains to the supervisory follow-up of the investigations conducted within TRIM. As in a regular IMI, the ECB determined the appropriate form of supervisory measures and timescales for next steps and drafted this in the form of a supervisory decision. Following this draft decision phase and the subsequent hearing period phase, the decision would be finalised and approved for sending to the institution.

110 See also the SSM Supervisory Manual on the ECB’s banking supervision website.

https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.supervisorymanual201803.en.pdf?42da4200dd38971a82c2d15b9ebc0e65


122

Assessment team composition

Regarding the team composition, each assessment team was led by an HoM who was always an NCA or ECB staff member (pursuant to Article 144(2) of the SSM Framework Regulation). On average at least half of the assessment team was composed of internal ECB or NCA staff (including the HoM). However, the part of the investigation dedicated to the IT and data quality review (as well as the rating assignment review) was in general an exception to this rule, as those teams were weighted more towards external staff. Additional exceptions were granted in a limited number of investigations where internal ECB or NCA resources were not sufficient to provide adequate staffing for the assessment team.

Chart A.1 Average resource allocation for TRIM investigations

(y-axis: average number of resources per investigation)

Source: ECB.

Supplementary information for Section 5.5 (Impact of TRIM supervisory decisions)

The purpose of this annex is to provide more detailed information on the impact of TRIM on the capital requirements of the institutions included in the exercise, in particular in terms of RWA (absolute and relative changes) and CET1 ratio impact. The impact expected from remediation of obligations or alignment with recommendations is not quantified.

Figure A.2 and Figure A.3 show the aggregated RWA and CET1 impact of supervisory decisions issued for the inspected models at SSM level.

3.93.1

2.20.8

4.9 5.9

5.3 7.6

0

1

2

3

4

5

6

7

8

9

10

Credit risk - models for retailand SME

Credit risk - models for LDP Market risk Counterparty credit risk

ExternalSSM


123

Figure A.2 Impact on RWA (EUR billions) and CET1 ratio of inspected models111 – SSM level

Impact on RWA of inspected models (aggregate RWA, EUR billions)

Impact on CET1 ratio (aggregate CET1 ratio impact, basis points)


The aggregated impact of limitations and material model changes approved with TRIM decisions led to an increase of about €275 billion (12%) in the RWA covered by the models assessed in the corresponding TRIM investigations. More than 90% of the increase in RWA was due to credit risk supervisory measures, as can be seen in Figure A.3 below. The increase in RWA reduced the CET1 capital ratio by 60 basis points (bps)112, mainly owing to the credit risk impacts (-56 bps). Both market risk and counterparty credit risk decreased by 2 bps.

111 The reference date can vary depending on the investigation considered. For credit risk missions, the

scope can vary depending on the risk parameter considered. 112 The CET1 capital ratio impacts are calculated at aggregated SSM level; differences might be noticed

from the evidence reported in the previous sections of the report (e.g. Section 5.5) where the median and the average impacts at individual institution level were illustrated.

+12%

+275

2,2482,523

Initial value Including impact of limitations and model changes

-39

-17

-2-2-60

CR LDPCR Retail and SMEMRCCR


124

Figure A.3 Impact on RWA (EUR billions) of inspected models – SSM level by risk type


The remaining part of this section provides an overview of the limitations and detailed information on the supervisory measures imposed in connection with the TRIM investigations.

779

188

1,144

106

219

RWA finalRWA initial RWA impact

2,248

275 2,523+12%

65

11

11

CR Retail and SMECR LDPCCRMR


125

Figure A.4 Overview of limitations – SSM level

Total number of decisions

Total number of limitations

Measures by type (percentages)

Source: ECB supervisory information. Notes: One decision might include more than one limitation and one limitation can include more than one supervisory measure. The category “Other limitations MR/IMM” includes: Floor at SA for MR and IMM; and Qualitative and other measures. The category “Other limitations CR” includes: Add-on/multiplier on RWA non-performing; Add-on/multiplier on LGD for defaulted assets and/or ELBE; and Qualitative and other measures.

Figure A.4 shows that almost 74% of the TRIM decisions contain limitations restricting or modifying the (permitted) use of a model to ensure an appropriate calculation of own funds requirements during the time the bank is remediating the shortcomings. It was also observed that:

• 22% of supervisory measures were imposed due to shortcomings related to PD: with almost 65% of PD multipliers between 1.1 and 1.2;

• 17% of supervisory measures were imposed due to shortcomings related to LGD: with more than 60% of LGD multipliers between 1.05 and 1.1;

186

67

253

Decisions with

limitations

Decisions without

limitations

54

125

319

CR LDP

CR Retail

and SME

121

19

MR

CCR

2%3%3%2%

6%

7%

6%

4%

17%

22%

17%

9%

Other limitations MR/IMMFloor to CCF/Add-on/multiplier on EAD/CCFFloor to FIRB LGDAdd-on/multiplier on IRCIncrease of the alpha multiplier for IMMOther limitations CRAdd-on/multiplier on RWA performing Floor to RWA (FIRB and SA)Change of multiplier for VaR/sVaRAdd-on/multiplier on LGD performingFloor to RWA for non-performing/ELBEAdd-on/multiplier on PD


126

• 17% of supervisory measures were imposed due to shortcomings related to floor to RWA for non-performing/ELBE.

Figure A.5 and Figure A.6 provide detailed information on the credit risk supervisory measures imposed in connection with Corporate-SME and Mortgages investigations and LDP investigations.113

Figure A.5 Supervisory measures by type – focus on Retail and SME portfolios


For the Retail and SME credit investigations it was observed that:

• more than 80% of the supervisory measures fall into just three categories: shortcomings related to PD, LGD and floor to RWA (FIRB or SA);

• almost 57% of PD multipliers are between 1.1 and 1.2, but 10% greater than 1.5;

• almost 89% of LGD multipliers are between 1.05 and 1.1.

113 Different multipliers defined within the same decision are counted individually for the computation of the

statistics.

25%

20%

11%

1% 1% 2% 1% 2%

Corporate - SMERetail - Other non-SMERetail - Other SMERetail - Qualifying revolvingRetail - Secured by real estate non-SMERetail - Secured by real estate SME

6%

1%1%2%

5%

1%

8%

1%

4%

1%1%

2%3%

1%1%2%

32%

28%

21%

6%

3% 2%1%

8%

Floor to RWA (FIRB or SA)

Add-on/multiplier

on LGD performing

Add-on/multiplier

on PD

Add-on/multiplier

on RWA performing

Floor to CCF/Add-

on/multiplier on EAD/CCF

Floor to RWA for defaulted

Floor to FIRB LGD

Other limitations


127

Figure A.6 Supervisory measures by type – focus on LDP


For the LDP credit investigations it was observed that:

• almost 70% of the supervisory measures fall into three categories: shortcomings related to PD, LGD and floor to RWA (FIRB or SA);

• almost 70% of PD multipliers are between 1.1 and 1.2, but almost 10% greater than 1.5;

• almost 45% of LGD multipliers are between 1.05 and 1.1, but almost 17% greater than 1.5.

In addition, for LDP there is a higher use of floor to LGD/RWA amount at FIRB level (36% of the decisions). This reflects:

• the lack of reliable quantification of deficiencies in order to support the calibration of a multiplier/add-on, generally owing to lack of appropriate data;

• the overall weaknesses and non-compliance of the modelling framework for LGD.

7%2%

14%

1% 2%4%

Corporates - OtherCorporates - Specialised lendingInstitutions

2%

2%

11%

5%

19%

2% 7%

17%15%

37%

1%3%

13%

7%

Floor to RWA (FIRB or SA)

Add-on/multiplier

on LGD performing

Add-on/multiplier

on PD

Add-on/multiplier

on RWA performing

Floor to CCF/Add-

on/multiplier on EAD/CCF

Floor to RWA for defaulted

Floor to FIRB LGD

Other limitations

8%

2%

7%

4%

7%

2%2%

© European Central Bank, 2021

Postal address 60640 Frankfurt am Main, Germany Telephone +49 69 1344 0 Website www.bankingsupervision.europa.eu

All rights reserved. Reproduction for educational and non-commercial purposes is permitted provided that the source is acknowledged.

For specific terminology please refer to the SSM glossary (available in English only).

http://www.bankingsupervision.europa.eu/

https://www.bankingsupervision.europa.eu/home/glossary/html/glossa.en.html

targeted review of internal models - europa

Documents