Tuesday, Feb 27, 2024
Supervisory Stress Testing: A Primer
Supervisory stress testing is a technique used to assess the robustness of financial institutions and the broader financial system to severe but plausible shocks. Since the Global Financial Crisis of 2008, it has emerged as an important tool for supervisors that complements other approaches and methods.
Stress testing is one of the more complex tasks undertaken by supervisory authorities. Primarily used to inform solvency and liquidity adequacy assessments, it is also a critical method for exploring emerging risks and the threats they pose to financial institutions.
This TC Note and accompanying podcast discusses the design, effective use, limitations, and interpretation of stress tests by financial supervisors. It explores a range of stress tests, from elementary, top-down applications to more sophisticated tests using scenarios applied bottom-up to multiple financial institutions.
Speaker:
R. Barry Johnston, Program Leader, Toronto Centre
Host:
Demet Çanakçı, Senior Program Director, Toronto Centre
Read the transcript here. Read their biographies here.
Listen to the Podcast:
Read the TC Note:
TABLE OF CONTENTS
What is supervisory stress testing?
Background on supervisory stress tests
Use of supervisory stress tests as a supervisory tool
Informing solvency and liquidity assessments
A source of supervisory information
Introducing a macroprudential perspective
A method for exploring emerging threats and risks
Design of supervisory stress tests
Stress testing adverse macroeconomic conditions
Scenario designed stress tests
Sensitivity analysis using idiosyncratic shocks
Further aspects of supervisory stress tests
Static or dynamic stress tests
Limitations of supervisory stress tests
Governance of supervisory stress tests
Communication of supervisory stress tests
Communication between the supervisor and financial institutions
Publication of supervisory stress tests
Copyright © Toronto Centre. All rights reserved.
Toronto Centre permits you to download, print, and use the content of this TC Note provided that: (i) such usage is not for any commercial purpose; (ii) you do not modify the content of this material; and (iii) you clearly and directly cite the content as belonging to Toronto Centre.
Except as provided above, the contents of this TC Note may not be transmitted, transcribed, reproduced, stored or translated into any other form without the prior written permission of Toronto Centre.
The information in this TC Note has been summarized and should not be regarded as complete or accurate in every detail.
SUPERVISORY STRESS TESTING: A PRIMER
Introduction1
Stress testing is a technique to test the resilience of financial institutions and the financial system to adverse economic and financial conditions. Stress testing can take many forms, including assessments by financial institutions of the resilience of a single financial position or a financial portfolio as part of their internal risk management, and assessments of the stability of the financial system by macro prudential authorities. This Toronto Centre Note focuses on stress testing by supervisory authorities - supervisory stress testing.
Post the global financial crisis (GFC) of 2008, supervisory stress testing has become an important supervisory tool. Supervisory stress testing complements other supervisory approaches and methods. It is primarily used by supervisory authorities for informing solvency and liquidity adequacy assessments. Supervisory stress testing also provides an important source of supervisory information, is a critical method for exploring emerging risks and the threats they pose to financial institutions and the financial system, and introduces a macroprudential element into micro prudential supervision.
The Note is outlined as follows. It first defines supervisory stress testing and how it differs from other types of stress testing in the financial system, and provides some historical background. The Note explains the uses of supervisory stress testing and how it complements other supervisory tools. It discusses the different approaches to the design and implementation of supervisory stress tests. Some further aspects of supervisory stress tests are then considered, including the limitations of the stress tests, communication of the stress tests, and governance of the stress testing process. The final section concludes.
What is supervisory stress testing?
Supervisory stress testing is defined here as:
- A stress test designed and carried out at the direction of the supervisory authority for supervisory purposes, possibly in collaboration with other authorities;
- under which multiple financial institutions are concurrently subjected to the same set of shocks or adverse scenarios; and
- the results are used to inform or further supervisory objectives, including polices, understanding risks and interventions.
Supervisory stress testing covers both stress tests in which the supervisor uses its own models and data to assess financial institutions, as well as stress tests in which the supervisor provides shocks, scenarios and guidance to the financial institutions, which then use their own models and data and report the results to the supervisor.
The stress tests are designed and directed by the supervisory authority for supervisory purposes. Supervisory stress testing differs from the internal stress tests conducted by financial institutions. The internal stress tests are designed and implemented by the financial institutions as part of good risk management practices.2 Undertaking stress tests may be mandated by the supervisor and overseen by them, but the stress tests are not designed by the supervisor and the shocks and scenarios used in the stress tests are not coordinated or conducted concurrently across financial institutions.
The supervisory stress tests also differ from those conducted by the macroprudential authority to assess financial stability, although there are overlaps and similarities between these types of stress tests. Macroprudential stress tests explore the threats to the financial system, while supervisory stress tests explore the threats to individual financial institutions. The stress tests overlap when they address the risks posed by systemically important financial institutions (SIFIs) since the risks to individual SIFIs also pose risks to the financial system. However, supervisory stress tests will often examine the risks to a broader range of financial institutions, systemic and non-systemic. The financial stability objectives may also be served through top-down stress testing (defined and discussed below), whereas this is less likely to be the case with supervisory stress tests, where the input of individual financial institutions in the stress testing is desirable if not essential.
The need for collaboration with other authorities in the design of the supervisory stress tests will depend on the topic. Stress tests that explore adverse macroeconomic or macro financial scenarios should preferably be coordinated with macroprudential authorities. The need for collaboration will be less critical in the case of supervisory stress tests that examine the sensitivity of balance sheets to specific types of risk. The need for collaboration may also be determined by technical considerations and the need to draw on the expertise of other authorities in implementing the stress test.
The stress test assumptions or scenarios are provided by the supervisory authority and applied to multiple financial institutions concurrently. Most often the tests are applied to the most important banks, but depending on the topic they may also applied to a wider range of banks and to other financial institutions such as insurance companies, pension funds and central counterparties.
The stress test is run concurrently, which means that each financial institution will be subject to the same set of shocks or adverse scenarios and the shocks are applied using consistent assumptions and on a consistent time frame. The concurrent nature of the stress test allows the supervisor to obtain an overview of how the scenario or shock will impact different financial institutions, and in a bottom-up stress test (defined below) to compare how different institutions assess the shock or stress scenario, allowing for peer review of practices. The concurrent stress test will also identify potential systemic implications of the shocks or scenarios, allowing the supervisory authority to factor these into the results.
The supervisory stress test informs or furthers supervisory objectives, including polices, understanding risks and interventions. Supervisory stress tests most often explore the solvency or liquidity of financial institutions and the resilience of capital or liquidity buffers to severe but plausible economic and financial events. “Passing” a supervisory stress test of solvency or liquidity may be made an explicit supervisory requirement. Supervisory stress tests of emerging risks, such as climate change and cyber risk, help to inform the supervisory responses to those risks. More generally, supervisory stress tests are a complement to other supervisory approaches in understanding risks and assessing the adequacy of capital, liquidity, and risk management frameworks.
Background on supervisory stress tests
Stress tests have for some time been used as a tool to assess financial stability, and were adopted in 1999 as one of the key elements of the IMF’s Financial Sector Assessment Program (FSAPs).3 While these stress tests were for the purpose of assessing financial stability, in the case of financial systems dominated by SIFIs they also evaluated the solvency of individual banks. However, the IMF stress tests were rarely used for supervisory purposes, as prior to the GFC supervisory frameworks were evaluated almost exclusively using assessments of compliance with international supervisory core principles.
Stress testing’s use as a supervisory tool largely dates from the GFC. The first substantial supervisory stress test was the US Federal Reserve’s Supervisory Capital Assessment Program (SCAP), run in early 2009 to help calibrate the financial position of the major US banks in the face of extreme financial uncertainty. During the GFC, traditional indicators of banking solvency, such as risk weighted capital adequacy, were unreliable. This made it difficult to distinguish between solvent and insolvent banks, and contributed to the severe market turbulence. The SCAP was designed to provide information on the underlying financial condition of the major US banks and their capacity to withstand a further downturn in economic activity. The publication of the results of the stress test reduced financial uncertainty and, combined with the US Treasury’s willingness to backstop the weaker banks with additional capital if needed, helped to stabilize US and global financial markets. The SCAP is described in more detail in the box below.
Box 1: US Supervisory Capital Assessment Program (SCAP)4
|
The SCAP was intended to measure the financial strength of the US’s 19 largest banks in early 2009. The financial crisis had left many banks and other financial institutions severely undercapitalized. However, given the enormous financial uncertainty created by the GFC the markets had lost confidence in traditional indicators of banking soundness such as regulatory capital ratios. The stress tests were intended to show how well the banking sector could withstand the impact of a major economic downturn and to clarify how much additional capital the banks might require. The SCAP involved several elements. First, each of the SCAP banks was asked to perform a capital adequacy stress test under two economic scenarios - baseline and more adverse - using specified assumptions for GDP growth, unemployment, and house prices. The baseline scenario reflected the consensus expectation among professional forecasters on the depth and duration of the recession. The more adverse scenario was designed to be severe but plausible, with a probability of roughly 10 to 15 percent that each of the macroeconomic variables could be worse than specified. The banks were asked to provide projections of losses and revenues under the two scenarios. Losses were to be projected over a two-year horizon for at least 12 separate categories of loans and a few other asset classes, using year-end 2008 financial statement data as a starting point. To guide the banks, supervisors provided indicative loss-rate ranges for the system, derived from analysis of the historical loss experience at large banks and quantitative models relating loan performance to macroeconomic variables. Banks were informed that loss estimates below the indicative range would be closely scrutinized. Second, the supervisory teams evaluated the banks' estimates to identify methodological weaknesses, missing information, overly optimistic assumptions, and other problems. Examiners had detailed conversations with bank managers, which led to numerous modifications of the banks' submissions. Supervisors then made judgmental adjustments to the banks' loss and revenue estimates based on sensitivity analyses performed by the firms, comparative analysis across the firms, and the supervisors' own judgments. Third, the supervisors supplemented these judgmental assessments with objective, model-based estimates for losses and revenues that could be applied on a consistent basis across the banks. Each participating institution was asked to supply, in a standardized format, detailed information that supervisors could use to estimate losses and revenues, such as details about loan characteristics. These data allowed supervisors to make consistent estimates using independently constructed models. Finally, supervisors systematically incorporated all these inputs into loss, revenue, and reserve estimates for each bank. These estimates were combined with information on existing reserves and capital to project capital buffers that the banks would need under the two scenarios. Supervisors released the methodology and assumptions underlying the stress test first and then, two weeks later, the results for individual institutions. The results showed that under the more adverse scenario, 10 of the 19 SCAP banks would need to raise a total of $75 billion in capital to have the capital buffers that were targeted under the SCAP. Subsequently, the Federal Reserve has used annual stress tests to evaluate plans by large banks to make capital distributions and approves these plans only for institutions that demonstrate sufficient financial strength under a severe stress scenario. |
Following the successful experience with the SCAP, supervisory stress testing became much more widely used. The European banking authorities adopted stress testing soon thereafter as a key supervisory tool, as did other national supervisory authorities.5 The European Central Bank (ECB) conducts several types of stress tests to assess how well banks can cope with financial and economic shocks.6 Every two years an EU-wide stress test is based on an adverse macroeconomic scenario. In years when there is no EU-wide stress test, the ECB tests significant institutions against a specific kind of shock.7 In Canada, the supervisory authority (OSFI) and the Bank of Canada conduct an annual joint exercise to stress test the major Canadian banks; similarly, the Bank of England conducts an annual supervisory stress testing exercise.8
While supervisory stress testing has largely been applied to the banking sector, its use has been extended to other sectors. In 2018 IOSCO issued guidance on the design and execution of supervisory stress tests for central counterparties (CCPs). The IOSCO framework is designed to help authorities better understand the macroprudential risks that could materialize if multiple CCPs were to face a common stress event.9 Supervisory stress testing is applied in the pension sector to assess the resilience of occupational pension funds to adverse market and economic conditions.10 Insurance supervisory stress tests explore the resilience of the insurance sector to climate change.11
Use of supervisory stress tests as a supervisory tool
Supervisory stress tests have several uses as a supervisory tool. They are primarily used by supervisory authorities for informing solvency and liquidity adequacy assessments. They are a source of supervisory information in the face of uncertainty, and are used to assess the risks posed by emerging and specific threats to financial institutions and the financial system, and in macroprudential analysis.
Informing solvency and liquidity assessments
The most frequent use of supervisory stress tests is as part of supervisory assessments of the solvency and liquidity of financial institutions. Financial institutions and supervisory authorities apply various techniques to manage and mitigate risks on financial institution balance sheets. For example, in managing risks in banks’ loan portfolios:
- Before granting the loan, the bank should assess the capacity of the borrower to repay the loan and reflect this in the terms of the loan: its maturity, the interest rate charged on the loan and the nature of the collateral pledged by the borrower against the loan.
- The bank should set aside provisions to reflect the expectation that some of the loans will not be repaid. Loan provisions are intended to cover the expected losses on the loan portfolio.
- The bank should hold capital against unexpected losses: the possibility that losses on the loan portfolio will exceed the provisions or expected losses. The minimum capital is set by regulatory standards (Pillar 1). The bank may also, or be required to, hold additional capital based on its own assessment of the size of unexpected losses, its internal capital adequacy assessment process (ICAAP), or the supervisory review of the risk profile (Pillar 2). The bank’s ICAAP should include internal stress testing.
- The bank may also be subject to additional macroprudential capital add-ons, including if applicable a charge for systemically important banks and a counter cyclical capital buffer reflecting the growth of credit relative to GDP.
Supervisory stress testing can be used to review the adequacy of these buffers in the face of severe but plausible shocks. A single factor stress-test (see discussion below) can explore the adequacy of provisions and is used as a basic stress test for banking systems in countries with less developed institutional structures. More sophisticated stress testing is used as part of the supervisory review process, including of banks’ internal stress tests. In some countries, the stress tests may be part of the process for setting Pillar 2 capital or calibrating the counter cyclical capital buffers. At least one country has a specific additional capital add on derived from the results of the stress test.12
The above examples relate to a bank’s capital buffers. Similar considerations apply to banks’ liquidity buffers. Supervisory stress tests of liquidity buffers include, for example, the adequacy of liquidity coverage ratios and the resilience of a bank’s funding to the risk of contagion in funding markets. Some other common uses of supervisory stress tests are as a complement to on and off-site supervision and to assess banks’ recovery plans (for both solvency and liquidity).13
A source of supervisory information
Supervisory stress tests can be an important source of supervisory information in the face of institutional, economic, or financial uncertainty. In periods of increased uncertainty about current or future events or markets conditions, traditional indicators and sources of information may become unreliable. Measures and indicators based on historical information may cease to be good predictors of the future.
Stress testing can help to identify and fill information gaps in periods of uncertainty. Stress tests achieve this by imposing an analytical discipline in exploring the question of how certain shocks or events will impact balance sheets and operations. Answering this question requires the identification of information on the nature of a financial institution’s exposures, and methodologies to assess how the shocks will impact operations and exposures.
The results of the stress testing exercise can be both qualitative – identifying information and knowledge gaps that need to be filled – and quantitative - developing best estimates of the impacts of shocks based on available information. For example, an outcome of some initial stress tests was to identify the difficulties in consolidating exposures across banking groups and the need to develop appropriate sources of information. Another example is the recent use of stress tests to explore the implications of climate change (see below).
When conducted as bottom-up stress tests (see below), a financial institution’s ability to deliver on the stress test provides important information on the institution’s internal governance and technical capability. The concurrent nature of the supervisory stress test allows for peer review of institutions, and identification of good practices and outliers.
Introducing a macroprudential perspective
Following the GFC it has been recognized that micro prudential authorities need to include a macroprudential or system wide perspective as part of their supervisory frameworks. Supervisory stress testing helps to achieve this.14
First, the development of the adverse economic and financial scenarios for the supervisory stress tests requires consideration of how adverse macro-economic and financial events will impact financial institutions – for example how an economic downturn would affect concentrations of exposures to common participants, common risk factors or common dependencies of central counterparties to liquidity providers. Supervisors will wish to implement scenarios that are challenging to the exposures of financial institutions and thus will need to understand the transmission channels through which macro-economic and financial events impact financial institutions’ exposures and operations.
Second, the supervisory stress tests allow for the introduction of systemic effects and macroprudential feed backs into the assessment of the financial condition of individual financial institutions. Individual financial institution stress tests consider exposures on the institution’s own balance sheet, but cannot take account of systemic effects resulting from weaknesses or failures that a supervisory stress test might identify in other financial institutions. By conducting stress tests across multiple institutions, the supervisory stress tests can identify system wide weaknesses and potential feedbacks between financial institutions that could amplify the losses in individual institutions and the financial system.15 The potential second round effects can be introduced by the supervisor to modify and elaborate the individual financial institution stress tests.
Third, the supervisory stress tests should involve coordination and collaboration with the macroprudential authorities. The macroprudential authorities can help identify the relevant economic and financial risks for the stress test; and the results of the stress test can help inform macroprudential policy by identifying systemic risks. The technical skills of a macroprudential authority may also complement those of the supervisor in the modelling of the economic and financial effects of scenarios used in the stress test.
A method for exploring emerging threats and risks
Supervisory stress testing is a tool for exploring the impact of emerging threats and risks. Typically, emerging financial risks pose new and uncertain challenges for financial institutions. A supervisory stress test provides a framework where the challenges posed by the emerging risks can be explored as a desk top exercise or simulation. Conducting desk top exercises is potentially a much less costly way of identifying shortcomings in risk management than when confronted with the actual materialization of the emerging threats.
The exploration of emerging threats or risks is usually undertaken in the form of a thematic stress test that explores a specific source of risk. The stress test can be designed either as a sensitivity analysis to gauge the impact of the emerging risks or derived from a scenario that models the emerging threats. The results of the stress tests can help to identify the potential materiality of the risks, as well as areas for further examination and follow up in understanding and mitigating the risks.
Design of supervisory stress tests
Stress testing quantifies the impact of severe but plausible negative shocks. This section discusses approaches and considerations in the design of the stress tests.
While supervisory stress tests are designed by the supervisor, a basic question is how should they be implemented, by the supervisor, the individual financial institutions, or a combination of both. A supervisor implemented stress test is referred to here as a “top-down” stress test: one that is performed by a supervisor using its own data and models. The supervisor’s “own” data will include those sourced from financial institutions as part of regular reporting or special requests, including for the purpose of the stress test, and general statistical sources. A financial institution performed stress test is referred to as a “bottom-up” stress test.16 A stress test combining data and models from supervisors and financial institutions is referred to as a “hybrid” stress test.
In a bottom-up supervisory stress test, individual financial institutions apply the stress assumptions provided by the supervisor. In a top-down approach, the stress assumptions are applied by the supervisor to the financial institutions’ balance sheets. A hybrid approach involves elements of both, for example the bottom-up results may be revised by the supervisor to reflect its own models of the impact of losses.
A bottom-up stress test is potentially richer, as it should incorporate each financial institution’s proprietary information on its exposures, hedges, and risk management strategies. For large and complex financial institutions, bottom-up stress tests are preferred. However, the consistency of the stress tests across financial institutions is harder to control in a bottom-up test than in a top-down test, and the results are more difficult to interpret across financial institutions because the specific stress testing techniques may vary between institutions. Validation of financial institutions’ data and models is an important part of any bottom-up stress test.
The supervisor may adopt a hybrid approach to enrich the stress test or to mange inconsistencies in stress testing methodologies used by individual financial institutions. For example, it may set a range for the loss parameters within which the bottom-up stress test should be performed, based on the supervisor’s own (top-down) modelling; or it may request modifications to the bottom-up stress tests to address outliers in stress testing results identified in a peer review. The supervisor may also modify bottom-up results to take account of systemic effects and feedback mechanisms not reflected in an individual institution’s stress tests.
Stress testing adverse macroeconomic conditions
The most frequently conducted supervisory stress test is of adverse economic and financial – macroeconomic - conditions. The reason this stress test is so important is the historical experience of financial institution losses and failures during economic downturns.
A supervisory stress test of adverse macroeconomic conditions can be designed either as a single/multiple factor stress test or as part of a stress scenario. The difference is that the former is designed based on ad hoc shocks/assumptions, while the latter is designed based on an articulated scenario.
In a single/multiple factor stress test, a single or a range of shocks are identified associated with adverse macroeconomic conditions. These shocks are then used to stress the financial institution’s balance sheet to determine how the capital (and/or liquidity) buffers will behave under the shock(s). The choice of the shocks reflects the risk factors on the balance sheet that are to be considered, for example, credit risk, market risk, interest rate risk, and liquidity risk.
Figure 1: Shock-based stress test
An example of a single factor adverse macroeconomic stress test of credit risk would be to explore the implications of a one notch downgrade in the performance of banks’ loan portfolios. Where banks are provisioning based on loan performance, the stress downgrade will require banks to set aside additional specific loan provisions which will impact their capital buffers. The resilience of the bank to a one notch downgrade is determined by whether the bank’s stressed capital buffer remains above a predetermined threshold (minimum capital requirement). Other examples of common single factor adverse macroeconomic stress tests explore the implications of a sharp rise in short-term interest rates, a narrowing or inversion of the interest rate yield curve, a sharp depreciation in the exchange rate, or a change in the withdrawal rate on deposits.17
A multiple factor stress test would explore the concurrent impact of two or several shocks on financial institutions’ balance sheets. For example, the stress test might combine the downgrade in the performance of the loan portfolio with a fall in the value of collateral held against the loans, or it might consider the combined effects of interest rate, yield curve and exchange rate changes. The more shocks that are introduced, the more complex the stress test, and the more it will take on the characteristics of a stress test that uses a scenario to justify the complexity of the shocks. For example, a stress test of a loss of confidence in the currency could entail a shock to interest rates, the exchange rate, and withdrawals of funds.
The supervisor will need to specify the precise single/multiple factor shocks to be stress tested and their magnitude. The size of the shocks may be determined based on historical experience, for example the stressed level of loan delinquencies could be calibrated using the last financial crisis. Calibrating the shocks using historical experience can help in justifying the size of the shocks and in interpreting the results. Supervisors may also deliberately design the shocks outside the historical experience to explore resilience to exceptional or emerging risks, especially in thematic stress tests (see below).
The nature and size of the shocks may be designed to reflect the phase of the economic cycle. At the peak of the cycle, when loan delinquencies are low and collateral values inflated, a severe stress test on loan losses and collateral values would be warranted. At the bottom of the cycle, the stress test might focus on the risks of additional loan losses, depressed credit demand and the impact of a low interest rate environment.
Single/multiple factor stress tests are most often applied in jurisdictions with weaker institutional and technical capacity. They can be used effectively in top-down stress tests to gain an initial assessment of the risk exposures of financial institutions, which can be followed up with more detailed analysis of the weakness identified with individual institutions.
Scenario designed stress tests
A scenario-based stress test differs from a single/multiple factor stress test in first specifying a stress scenario and then deriving the size of shocks to be stress tested by modelling the implications of the scenario on economic and financial variables. The economic and financial variables derived consistent with the scenario are used to stress financial institutions and to derive the impact on their capital (and liquidity) buffers.
Figure 2: Scenario-based stress test
A stress test based on a macroeconomic scenario begins with the specification of the scenario. The design of the scenario usually reflects assessments of current macroeconomic or financial sector vulnerabilities confronting the financial sector. The issues of concern might, for example, include the risk of a major economic downturn due to a crash in the housing market, spill over effects from financial failures in important countries in the region, or the impact of geopolitical events or a pandemic.
The macroeconomic scenario could explore a single major source of economic vulnerability or several concurrent vulnerabilities. An example of the former is the Bank of Canada 2022 stress test which explored a global risk scenario in which economic activity contracts sharply over an extended period caused by a new wave of the COVID 19 pandemic.18 An example of the latter is the Bank of England annual supervisory stress test which explores a range of tail risks confronting the UK economy.19 The supervisor should consult with the macroprudential authority in identifying vulnerabilities for designing the scenario.20
The macroeconomic consequences of the scenario are modelled to derive the implications for the macroeconomy and key economic and financial indicators such as GDP, unemployment, interest rates, inflation etc. The modelling may involve the application of large macroeconomic models, which typically are developed and maintained by central banks and ministries of finance for macroeconomic forecasting.21 The supervisor may seek to collaborate with these authorities in deriving paths for economic variables consistent with the scenario. These authorities may also be interested in the outcome of the stress test and what it may indicate for the risks to the economy and financial stability.22 If collaboration is not forthcoming, and the supervisor does not have its own expertise in macro economic modelling, it may have to turn to outside experts or consultants to develop the economic projections.
The macroeconomic and financial sector indicators derived consistent with the scenario provide the inputs for the stress test. These indicators are mapped to the balance sheets of financial institutions to derive estimates of losses. The supervisor will need to specify what elements of the balance sheet are to be subject to the stress. This will normally include the credit and market risk exposures and - depending on the objectives of the stress test and the institutions being stress tested - liability exposures and funding risks.23
In a bottom-up stress test, the financial institutions would be responsible for the mapping of the economic indicators to their balance sheets. For example, if as part of its internal risk management the financial institution has developed models of how GDP and unemployment impact probabilities of default (PD) and loss given default (LGD) on various categories of loans, it can apply these models for the stress test. If such models do not exist then the financial institution may either have to develop them or use qualitative judgements to estimate the losses. In a top-down stress test, the supervisory authority would apply its own models or qualitative judgement to evaluate the losses.
A hybrid approach could employ various techniques. For example, the supervisor could use its models to calculate the PDs and LGDs resulting from the scenario and then request the individual financial institutions to apply them to their balance sheets;24 or the supervisor could leave it to the financial institutions to calculate the PDs and LGDs, but indicate what would be acceptable ranges.25
The stress test results may need to be modified to reflect systemic effects and macro financial feedback mechanisms as these may not be reflected in the projections of macro financial indicators used to calibrate the stress tests. For example, if the stress test identifies weaknesses in the capital position of a SIFI, this could have knock on effects in the financial system and economy. This might require adjusting the severity of the economic scenario or incorporating additional losses on assets or stresses in funding markets. The adjustments could be made judgementally or modelled.26
Supervisory stress testing can be used as a tool to explore the consequences of thematic risks, such as interest rate risks, liquidity risk, cyber risks, and climate change. Stress tests of thematic risks examine a specific source of risk and may explore the impact on part of financial institutions’ balance sheets or operations. They differ from the stress tests of adverse economic and financial conditions which are broader based. Thematic stress tests may be designed either as sensitivity analysis to idiosyncratic shocks or derived from scenarios.
Sensitivity analysis using idiosyncratic shocks
Sensitivity analysis using idiosyncratic shocks is intended to test the resilience of financial institutions to specific risks. The magnitude of the shocks explored can be varied as part of the stress test to calibrate the impacts under increasingly severe conditions, and hence to judge the sensitivity and resilience of financial institutions. The shocks may range from adverse to extreme, with the size of the shock increasing in intensity, and with the extreme shocks potentially larger than historical experience to test resilience to emerging risks or exceptional events.
The impact of the shocks is assessed under criteria provided by the supervisor such as the duration of the shock, the balance sheet items subjected to the shock, and permitted responses (“management actions”) of financial institutions to the shock. For example, a thematic stress test of liquidity risk could specify the size of liquidity outflows in increasing intensity, the period over which the outflows persist, and the assumptions to be applied in terms of accessing interbank funding or reducing lending in response to the outflow.27
The size of idiosyncratic shocks used for thematic stress tests may be derived from historical experience or determined judgementally. An example of a thematic stress test using idiosyncratic shocks was conducted by the ECB to evaluate interest rate risks on Euro area banks’ banking books.28 The ECB stress test used interest rate shocks prescribed by the Basel Committee derived from historical experience.29 The ECB noted, however, that the shocks were considered hypothetical and were not meant to forecast the future development of interest rates. Rather they were intended to identify potential vulnerabilities in banks’ banking books by capturing various changes in the level and shape of the interest rate curve, and to provide the supervisors with information on how the economic value of the banking book and net interest income projections would change under each shock. The shocks were therefore applied in the spirit of a sensitivity analysis.30
The most developed scenario based thematic stress tests explore the effects of climate change for financial institution balance sheets. Climate scenarios combine knowledge of the atmospheric and natural environment with that of the social and economic environment to project the impact of climate change on economic activity. The Network on the Greening of the Financial System (NGFS) has developed climate scenarios to provide central banks and supervisors with a common starting point for analyzing climate risks under different future climate pathways. The NGFS scenarios have been used widely by central banks and supervisors in their climate stress tests. The use and design of climate scenarios and their application in climate stress testing is discussed in detail in Toronto Centre (2023).
Another example of scenario based thematic stress testing is the ECB cyber resilience stress test in 2024. The exercise assessed how banks respond to and recover from a cyber attack, as opposed to simply looking at their ability to prevent it. Under the stress test scenario, a cyber attack successfully disrupts banks’ daily business operations. Banks then have to test their response and recovery measures, including activating emergency procedures and contingency plans and restoring normal operations. Supervisors will then assess the extent to which banks can cope under such a scenario. This is a predominantly qualitative exercise and the insights gained will be used for wider supervisory assessments and supervisors will discuss the findings and lessons learned with each bank.31
Further aspects of supervisory stress tests
This section discusses some further aspects in the design, implementation, and communication of supervisory stress tests, including the limitations of the stress tests.
Static or dynamic stress tests
Stress tests can be either static or dynamic. A static stress test examines the impact on the balance sheets of financial institutions at a point in time, and how losses will materialize over a future time-period without considering corrective management actions. The future losses are projected onto the existing capital/liquidity positions to assess the capacity of the financial institutions to absorb the losses.
A dynamic stress test examines how financial institutions’ portfolios and losses will evolve over a period and allows for management actions in response to the shock, such as raising additional capital, adjusting the portfolio, or implementing cost-cutting measures. The losses and the capacity of financial institutions to absorb the losses are assessed at a future point in time, taking account of management actions.
The choice between static and dynamic stress tests will reflect the objectives and design of the stress test. A static stress test is more extreme than a dynamic test since it ignores actions which financial institutions would typically hope to take if the adverse conditions were to materialize. Single/multiple factor and thematic idiosyncratic stress tests are usually designed as static in nature and most scenario-based stress tests are also static in design.
A dynamic stress test will more clearly articulate management responses, and may provide additional information on the approaches to risk management and governance in financial institutions. A dynamic stress test may be preferred when exploring risk over an extended period. For example, some climate risk stress tests are dynamic in nature. A static stress test is technically easier to implement, and potential management responses to the shocks can be discussed with financial institutions as part of a static stress test.
The design of supervisory stress tests should reflect their costs and benefits. In view of the resource costs, some degree of proportionality should be applied in designing the stress tests to take account of differences in the systemic importance of financial institutions.
Systemic financial institutions should be subject to the full rigor of a bottom-up stress test. As these institutions are often more complex, there are potentially significant benefits from conducting bottom-up stress testing in identifying balance sheet exposures. A less burdensome approach can be adopted for non-systemic institutions, including less granular forms of data collection or simplified stress test modelling approaches. Top-down stress testing combined with additional information requests on specific exposures and potential management responses may be sufficient in many cases.
In some countries, an explicit differentiation is made between the stress testing requirements for SIFIs and non-SIFIs. For example, in the US only the largest and most complex firms are subject to the annual quantitative stress testing exercise for the purposes of approving capital distributions. In the euro area, the ECB stress test is limited to designated Significant Institutions.
Limitations of supervisory stress tests
Supervisory stress testing is only one element of a robust supervisory framework. Supervisory stress testing complements other supervisory tools, such as capital rules, ICAAPs and ORSAs, but does not substitute for them. The stress test can explore the adequacy of capital and liquidity buffers, but rules and procedures are still needed for the establishment of the buffers in the first place.
Supervisory stress tests are only as good as the data and methodologies used, and assumptions made, in producing them. Supervisory stress testing is a complex exercise involving several steps: deciding what to stress test in the design of the scenario or the nature and size of the shocks to be explored; translating the scenarios to macroeconomic and other variables; mapping the variables and shocks to financial institution balance sheets; and deciding on what elements of financial institutions’ operations can or cannot respond to the shocks. The results of the stress test reflect the data, assumptions and estimations made at each stage in the process.
The interpretation and use of stress tests results needs to reflect their design. The purpose of the stress test needs to be clearly articulated and the use of the results should reflect those objectives. For example, some pilot stress tests of the effects of climate change have been designed to solicit information about data and methodological shortcomings in the assessment of climate risks. The results of these pilot stress tests can only provide first estimates of the potential impact of climate change on capital buffers.
The degree of comfort from “passing” supervisory stress tests should always be circumscribed. In addition to the potential weaknesses in the stress test assumptions, data and methodologies, financial systems have been subject to “black swan” events, which have fallen outside the range of the severe but plausible scenarios used in stress tests.32 This will continue to be the case.
Governance of supervisory stress tests
The governance structures supporting supervisory stress testing are a critical part of any supervisory framework. Governance includes the articulation of objectives and scope, clarity on internal and external roles and responsibilities, the structure and resourcing for stress testing frameworks, and the validation and challenge of frameworks and results.
Stress testing exercises are one of the most complex activities conducted by supervisory authorities and they therefore require significant planning. Resources, roles, and responsibilities need to be identified and assigned across the organization, and between authorities where relevant. Where multiple teams or authorities are involved in conducting the stress test, appropriate permissions and memorandum of understanding may need to be established for sharing of information and results of the stress tests.
Supervisory stress testing exercises require specialized technical expertise. Obtaining those technical resources is one of the biggest challenges associated with running a successful supervisory stress test. Skills needed typically include risk specialists, modelling experts and examiners. As noted above, collaboration with other authorities specialized in macroprudential policy and macroeconomic modeling will complement the skills and reduce the resource burden on the supervisors.
Supervisory guidance should be issued on the conduct of the stress test. The guidance should define the different actions to be taken by each of the stakeholders and timelines for the process. Separate guidance should cover: (1) the internal (and if applicable cross authorities) assignment of responsibilities; and (2) the instructions to the financial institutions for running the stress test. The latter should include details about the scenarios and shocks for the stress test; methodologies to be applied in implementation; and templates for collecting data and reporting results.
Authorities and financial institutions need to have validation processes in place to assess the soundness of the data inputs, outputs, models, and approaches used to implement the stress test. A sound validation framework is critical to ensure the credibility and usefulness of the stress testing framework. The validation approaches that supervisory authorities can use include: (i) consideration of whether the results appear to be reasonable (for example, comparison of results derived from bottom-up and top-down assessments); (ii) peer comparison of methodologies and results across financial institutions; (iii) review of financial institutions’ internal governance and quality control; and (iv) expert review of the appropriateness of models and data.
Communication of supervisory stress tests
Communication of the detailed parameters of a stress test (for example, scenarios and supervisory expectations) and the results of stress testing assessments is a critical component of a successful supervisory stress testing exercise.
Communication between the supervisor and financial institutions
Communication between authorities and participating financial institutions in a bottom-up or hybrid stress test includes: (1) in advance of the stress test – providing detailed instructions prior to the exercise; (2) as the stress test is being run – for example to answer questions on the application of the methodology; and (3) feedback when the assessments are complete. The latter would include quantitative results if these have been modified by the supervisor and the reasons for modifications; an assessment of the data and methodologies applied; risk identification and management; and governance capabilities in carrying out the stress testing.
Communication with the stressed financial institutions in a top-down stress test should include: (1) the objectives of the stress test; (2) the details of the shocks/scenario and the methodologies to be applied; and (3) the quantitative results. The stressed financial institutions should have an opportunity to comment on the use of data and methodologies applied.
Depending on the purpose of the stress test, the communications with the stressed financial institutions should indicate the follow up supervisory action from the stress tests. Considering the amount of financial institution and supervisory resources that are involved in conducting the supervisory stress tests, the stress testing exercises should be followed up with robust conversations between the supervisor and the financial institutions on the results of the stress test. The discussions should focus on data and methodologies as well as the specific results. Types of follow up actions may include the need for additional data collection, revisions to internal stress testing methodologies, modifications to governance and management processes, proposals for future stress tests, reviews of internal risk management approaches, ICAAPs and ORSAs, and revisions to capital and liquidity buffers.
Publication of supervisory stress tests
Publication of the high-level methodology and scenario design features and a summary of the results is a widespread practice in supervisory stress testing. A smaller number of authorities publish the stress test results for individual financial institutions. Publication of stress testing objectives, scenarios, methodologies, and results allows external observers to judge the resilience of the institutions covered by the stress test to the various risks incorporated, and improves the accountability of the relevant stress-testing authority.
When publishing the stress tests:
- The objectives and role of supervisory stress tests as part of the overall supervisory framework should be explained in public statements. This will help ensure that the public and the markets develop an understanding of the purposes of the stress tests and how to interpret the results. If multiple authorities are involved, public statements should be coordinated across the authorities.
- The scenario design should be clearly articulated in the public description of the stress test. The credibility of the supervisory authority may be judged by the nature of the stress test: whether the scenario or shock(s) are considered appropriate and severe. Too severe a stress test has a potential to undermine credibility as can an inadequately severe stress test. Even if the hypothetical nature of the scenario is emphasized, its nature and severity may create expectations of potential risk as the stress test gives investors another source of information about the risks facing financial institutions.
- The publication of individual financial institution stress test results should be accompanied by explanations of the use and implications of the results. This is especially important when the stress tests identify weaknesses, so the stress tests do not undermine confidence in financial institutions. In such cases, the remedial actions that are being taken to address the weaknesses should be included as part of the reporting of the results. For example, financial institutions may take pre-emptive action to strengthen their capital position ahead of the release of stress test results, and such actions should be reported along with the results.
Supervisory stress testing is a technique to assess the robustness of financial institutions to severe but plausible shocks. Following the GFC it has become an important supervisory tool that complements other supervisory approaches and methods.
Supervisory stress testing is primarily used for informing solvency and liquidity adequacy assessments. It is also an important source of supervisory information and a critical method for exploring emerging risks, and introduces a macroprudential perspective into micro prudential supervision.
Various approaches can be adopted in designing the stress tests. At a basic level they can take the form of a single or multiple factor top-down stress test, that can be applied in jurisdictions with weak institutional and technical capacity. More sophisticated stress tests can be designed using scenarios that capture the major threats to financial institutions, and are applied as bottom-up stress tests in more advanced economies. Thematic stress tests can be used to explore the sensitivity to, and threats posed by, emerging risks.
Supervisory stress testing requires a robust governance framework as it is one of the more complex tasks undertaken by supervisory authorities. In view of the resources that are involved, follow up on the results is a critical part of any successful stress testing exercise.
Anand et al. Stress Testing the Canadian Banking System: A System-Wide Approach. June 2014.
Bank of Canada. How well can large banks in Canada withstand a severe economic downturn? May 2022.
Bank of England. Stress testing of banks: an introduction. September 2016.
Bank of England. Stress testing the UK banking system: key elements of the 2022/23 annual cyclical scenario. September 2022.
Basel Committee on Banking Supervision. Interest rate risk in the banking book. April 2016.
Basel Committee on Banking Supervision. Supervisory and bank stress testing: range of practices. December 2017.
California Department of Insurance. Scenario Analysis (ca.gov) May 2018.
Cihak, M. Introduction to Applied Stress Testing. March 2007.
European Central Bank. ECB sensitivity analysis of IRRBB – stress test 2017. February 2017.
European Central Bank. ECB tests banks’ sensitivity to liquidity risk. February 2019.
European Central Bank. Stress tests. January 2024.
European Insurance and Occupational Pensions Authority. 2022 Occupational pensions stress test. December 2022.
Financial Stability Institute. Stress-testing banks - a comparative analysis. November 2018.
International Actuarial Association. Stress Testing and Scenario Analysis. July 2013.
International Organization of Securities Commissions. Supervisory Stress testing of Central Counterparties. April 2018.
Tarullo, D.K. Lessons from the Crisis Stress Tests. March 2010.
Toronto Centre. Integrating Micro prudential Supervision with Macroprudential policy. May 2021.
Toronto Centre. Introduction for supervisors to scenarios and stress tests of climate change risks. May 2023.
Toronto Centre. Supervision of Stress Testing by Financial Institutions. March 2024.
1This note was prepared by R. Barry Johnston. Please address any questions about this Note to This email address is being protected from spambots. You need JavaScript enabled to view it.
2Toronto Centre (2024) explores the use of stress testing by individual financial institutions and the role of supervisors in assessing these stress tests.
3Some national authorities, including Finland and Sweden, included macro stress testing even earlier as part of their financial stability reports. See also Financial Stability Institute (2018).
4The discussion of SCAP is largely taken from Tarullo (2010).
5For example, Canada in 2013 and the UK in 2014.
6ECB (2024).
72024: Cyber risk; 2022: Climate risk; 2019: Liquidity risk; 2017: interest rate risk.
8See Basel Committee (2017) for a survey of the use of stress tests by supervisory authorities.
9IOSCO (2018).
10See, for example, EIOPA (2022).
11See, for example, California Department of Insurance (2018).
12See Basel Committee (2017) for a survey of the use of supervisory stress tests in setting banks’ capital adequacy requirements.
13For examples of supervisory stress tests used in other sectors, see references above.
14Toronto Centre (2021).
15See, for example, IOSCO (2018).
16“Top-down” and “bottom-up” are sometimes used to refer to the nature of the data inputs for models used in the stress test rather than the institution which conducts the stress test. The former refers to the use of aggregated data and models and the latter disaggregated (institution specific) data and models. This terminology is not followed in this Note.
17See, for example, Cihak (2007), and IOSCO (2018) for shocks applicable for CCPs.
18Bank of Canada (2022).
19Bank of England (2022).
20See Toronto Centre (2021).
21Various modelling techniques involving differing degrees of aggregation and sectoral complexity may be applied. See FSI (2018).
22For example, in Canada and Germany the supervisor and central bank collaborate closely on the stress tests.
23Insurance and pension fund stress tests include liability as well as asset exposures.
24This approach has been used by the supervisory authority in Mexico. See Basel Committee (2017).
25The US SCAP is an example, see Tarullo (2010).
26See Anand et al (2014) for a modelling approach used by the Bank of Canada to incorporate second round systemic liquidity risks in the stress tests.
27See for example ECB (2019).
28ECB (2017).
29Basel Committee (2016), Appendix 2. The shocks were derived from interest rate data covering a sixteen-year period.
30See ECB (2017).
31ECB (2024).
32For example, the stress tests conducted prior to 2008 did not anticipate the GFC; and the stress testing of the impact of a pandemic on the financial system was not considered before the onset of COVID 19 in 2020.