Friday, Mar 01, 2024
Supervision of Stress Testing by Financial Institutions
Stress testing should be a critical element of risk management for most financial institutions. It should alert boards and senior management to potential adverse outcomes related to a broad range of risks and vulnerabilities, and identify potential losses, liquidity needs, and operational responses should adverse shocks occur. Supervisors should have a strong interest in stress testing by financial institutions.
This TC Note and accompanying podcast discusses the need for financial institutions to conduct stress testing, what types of stress tests and scenario analyses are used, possible adverse outcomes, key areas in stress testing supervision, and the possible uses of the results by financial institutions.
Speaker:
Clive Briault, Chair, Banking Advisory Board, Toronto Centre
Host:
Demet Çanakçı, Senior Program Director, Toronto Centre
Read the transcript here. Read their biographies here.
Listen to the Podcast:
Read the TC Note:
TABLE OF CONTENTS
Why should financial institutions run stress tests?
Adequacy of financial resources
What can go wrong in stress testing by financial institutions?
Off-site and on-site supervision
A financial institution’s high level approach to stress testing
Supervisory response and intervention
Copyright © Toronto Centre. All rights reserved.
Toronto Centre permits you to download, print, and use the content of this TC Note provided that: (i) such usage is not for any commercial purpose; (ii) you do not modify the content of this material; and (iii) you clearly and directly cite the content as belonging to Toronto Centre.
Except as provided above, the contents of this TC Note may not be transmitted, transcribed, reproduced, stored or translated into any other form without the prior written permission of Toronto Centre.
The information in this TC Note has been summarized and should not be regarded as complete or accurate in every detail.
SUPERVISION OF STRESS TESTING BY FINANCIAL INSTITUTIONS
Introduction1
Stress testing should be a critical element of risk management for most financial institutions. It should alert the boards and senior management of financial institutions to potential adverse outcomes related to a broad range of risks and vulnerabilities; and it should help to identify potential losses, liquidity needs and operational responses should adverse shocks occur.
Supervisors should, in turn, have a strong interest in stress testing by financial institutions. This should be a key element in the supervisory assessment of the adequacy of a financial institution’s capital, solvency and liquidity, and the quality of a financial institution’s governance and risk management.
This Toronto Centre Note focuses on how supervisors should review a financial institution’s stress testing; and how this might feed into supervisory interventions. It is relevant for all types of financial institution, across all sectors.
Other Toronto Centre Notes2 have focused on stress testing run by supervisory authorities and macroprudential authorities. Many of the technical issues are similar whoever runs the stress tests, but there are important additional supervisory perspectives where a financial institution runs its own stress tests.
This Note discusses:
- the reasons why financial institutions should run their own stress tests;
- the international standards relating to stress tests run by financial institutions;
- some lessons from the Global Financial Crisis and the bank failures in March 2023;
- how supervisors can assess the stress tests run by financial institutions; and
- supervisory intervention.
Why should financial institutions run stress tests?
Stress testing is an integral part of good risk management. It is a key element in how a financial institution understands the risks it is running; sets its risk appetite; and determines its financial resilience (how much capital, solvency and liquidity it should hold) and operational resilience.3 Stress testing should therefore be viewed as a fundamental element in a financial institution’s overall risk management framework.
A financial institution needs to consider a wide range of possible outcomes that may affect its current and future financial positions, covering both sides of its balance sheet (and its off-balance sheet positions). Stress tests are a necessary risk management tool for a financial institution to help it to quantify the potential impact of different stress scenarios on its current and future financial positions; to assess whether it could absorb possible losses and meet liquidity needs that could occur under various scenarios and stresses; and thereby to identify better the risks it is taking.
Different financial institutions will face different risks. A financial institution’s stress tests should reflect the specific credit, insurance, market, interest rate, liquidity and operational risks run by that financial institution. For example, an insurer specialising in motor insurance, or a bank specialising in lending to the agricultural sector, will face different risks to an insurer specialising in building and contents insurance, or a bank specialising in mortgage lending.
Stress tests are not intended to be a central forecast of what will happen, but rather a tool for identifying what might happen in a particular stress scenario. Stress tests should be plausible and severe, not just mildly uncomfortable possibilities.
Stress testing can also be used to assess the potential impact of different levels of stress on a financial institution. For example, a financial institution could use stress testing to assess the impact of a range of macroeconomic scenarios, with different severities of declines in GDP and increases in unemployment, on a bank’s loan book; of a range of climate-related scenarios on insurance claims; or of a range of equity price reductions on an insurer’s or a position-raking securities firm’s assets.
A financial institution should then be in a better position to manage these risks, including by setting a risk appetite and corresponding limits and controls to maintain risks at levels consistent with this risk appetite, and by considering alternative strategies for mitigating its risks.
As part of good risk management, it is important that stress testing contributes to the understanding that the board and senior management have of the risks facing a financial institution. This requires the board and senior management to contribute to discussions of the stress tests that the financial institution should run; to understand the assumptions underlying the stress testing; to understand and discuss the results of stress testing; and to decide what actions the financial institution should take in response to these results.
Adequacy of financial resources
Stress tests can help a financial institution to maintain adequate financial resources. Stress tests should be a key element of a financial institution’s own assessment of the adequacy of its financial resources. This should be reflected in a bank’s internal capital adequacy assessment process (ICAAP) and an insurer’s own risk and solvency assessment (ORSA).4
However, this cannot be an automatic calculation. A financial institution can decide to hold financial resources that are adequate to enable it to cope with a given level of stress, while recognising that these resources might not be adequate if the stress was significantly more severe.
There is clear linkage between a financial institution’s scenario analysis and stress testing and its recovery plan.5
A recovery plan should be based on a range of clearly articulated, severe but plausible, firm-specific, market-wide and systemic stress scenarios, and combinations of these. The scenarios should cover both fast-moving and slow-moving events. These scenarios should include, but not be limited to, the scenarios used by a financial institution for its stress testing. For example, a life insurer should consider scenarios such as pandemics; a significant increase in longevity following a medical breakthrough; a mass lapse of policies; the failure of significant counterparties; a major cyber-security breach; significant falls in financial markets; and significant changes in the interest rate environment.
Financial institutions should analyse the potential impact of these scenarios on their profitability, capital, and liquidity; credit rating and cost of raising funding; external counterparties; operational capacity; material legal entities; core business lines; critical functions and critical services; and group-wide position.
Financial institutions should then develop “recovery actions” that could enable them to recover from such scenarios.
As discussed in other Toronto Centre Notes6, various types of stress testing can be undertaken.
The simplest is a sensitivity test, which estimates the impact of one or more moves in a particular risk factor, or a small number of risk factors, on the future financial condition of a financial institution.
For example, a financial institution could simply assume that equity prices will fall by 20 per cent; or that interest rates will increase by 3 percentage points; or that motor insurance claims will increase by 30 percent; or that residential property prices will fall by 10 per cent; and apply one or more of these changes to relevant balance sheet items.
Sensitivity tests can provide a fast, and relatively easy to implement, initial assessment of portfolio sensitivity to a given risk factor and identify certain risk concentrations. However, they do not relate these “single factor” shocks to an underlying scenario, event or real-world outcome.
At the other end of the spectrum, complex scenarios could be constructed involving simultaneous movements in a wide range of risk factors. There are two basic types of scenarios: historical and hypothetical.
Historical scenarios reflect changes in risk factors based on movements that have occurred in previous historical episodes. They are often used to develop macroeconomic scenarios, based on shocks that have been observed in previous economic recessions.
Hypothetical scenarios are based on shocks that are thought to be severe and plausible, but have not yet occurred in a country.7 For example, this approach could be used to develop climate-related scenarios. A financial institution could then model8 the ways in which these scenarios might be translated into stresses on a wide range of risk factors, which in turn might have an impact on its financial position.
Each type of scenario has its benefits, and large financial institutions might want to use both approaches, depending on the nature of the risks they face.
For example, a climate-related scenario could provide a starting point in terms of physical and transition risks.9 Various techniques and assumptions could then be used to model the relevant transmission channels (including regional and sector-specific impacts) by which these physical and transition risks could feed through to stresses on a financial institution’s financial position. This could be through the impact of physical or transition risks on macroeconomic variables, and through more direct channels such as the impact of climate events on insurance claims or on the impairment of bank lending to the fossil fuels, agriculture or tourism sectors.
International standard setters such as the Basel Committee and the International Association of Insurance Supervisors (IAIS) refer to stress testing as:
“integral to firms’ risk management, in that it alerts the firm’s management to unexpected adverse outcomes arising from a wide range of risks, and provides an indication to firms (and their supervisors) of the financial resources that might be needed to absorb losses should large shocks occur.”10
“a necessary tool in assisting a firm to manage its risks and maintain adequate financial resources to deal with those risks.”
“used to identify and quantify the impact of different stress scenarios on a firm’s future financial position.”
The importance of stress testing by banks and insurers is reflected in the principles and guidance issued by the Basel Committee (2018) and the IAIS (2003), and in the many references to stress testing in the Basel and IAIS Core Principles.11 These cover many aspects of stress testing by financial institutions, including:
Proportionality - the principles and guidance should be applied on a proportionate basis, depending on the size, complexity and risk profile of the financial institution.
“for internationally active firms, stress testing is expected to be embedded as a critical component of sound risk management.”
“smaller firms can benefit from considering in a structured way the potential impact of adverse scenarios on their business, even if they are not using a formal stress testing framework but are instead using simpler methods.”
Scope and objectives – stress testing should be used to identify potential losses and liquidity needs under adverse circumstances, and as an integral element of risk management.
“a firm should have in place a sound firm-wide risk management framework that enables it to recognise all material risks.”
“the risk identification process should include a comprehensive assessment of risks, which may include those deriving from both on- and off-balance sheet exposures, earnings vulnerabilities, operational risks, and other factors that could affect the solvency or liquidity position of the firm.”
“a firm needs to consider a wide range of possible outcomes that may affect its current and expected future financial position …. to determine whether it is financially flexible to absorb possible losses that could occur under various scenarios.”
“all the effects of stress testing, both direct and indirect, on both sides of the balance sheet, should be taken into account.”
Relevance – scenarios and stress tests should be appropriate to a firm’s risk profile.
“firms should ensure that scenarios are tailored to their businesses and address their firm-specific vulnerabilities.”
Severe and plausible – scenarios and stresses should be sufficiently severe and plausible to provide a meaningful test of a firm’s financial and operational resilience.
“stress testing should address significant adverse threats to the future financial condition of the firm, rather than just mildly uncomfortable possibilities, so as to truly test the firm’s exposure and the sufficiency of its capital and liquidity.”
Governance – the board should be actively involved in scenario analysis and stress testing.
“the firm’s board of directors should have the ultimate responsibility for the overall stress testing framework, including the oversight of the framework. The development and implementation of the stress testing framework may be delegated to senior management or a stress testing committee.”
“the board is expected to have an understanding of the material aspects of the stress testing framework that enables it to actively engage in discussions with senior management or senior experts that are responsible for stress testing, and to challenge key modelling assumptions, the scenario selection, and the assumptions underlying the stress tests.”
Organisational structures, policies and processes – firms should have a clear internal framework and internal controls for running stress tests.
“roles and responsibilities should be specified for all aspects of the stress testing framework, including scenario development and approval, model development and validation, reporting and challenge of results, and the use of stress test outputs. The roles of the second (risk management and compliance) and third (internal audit) lines of defence should be specified.”
Frequency – firms should run stress tests on a regular (usually annual) basis, and in response to changes in strategy, business model, and external conditions.
Resources and expertise – firms should have sufficient resources and expertise to run the types of scenario analysis and stress testing that are appropriate for them, given their size, complexity and risk profile.
Methodology – firms should have the data, modelling techniques, model development and model validation required to support a programme of scenario analysis and stress testing.
“in order for risks to be identified and the results of stress tests to be reliable, the data used should be accurate and complete, and available at a sufficiently granular level and in a timely manner.”
“firms should have in place a robust data infrastructure capable of retrieving, processing, and reporting information used in stress tests to ensure that the information is of adequate quality to meet the objectives of the stress testing framework.”
“modelling choices and calibration decisions should consider the interactions between different risk types, as well as the linkages among models. Links between solvency and liquidity stresses should be considered. Firm-wide stress testing should include all material risks and a sound aggregation of results.”
“stress tests should examine the effects and impact that different time horizons will have on business plans, strategic risks and future operating requirements. The time horizon needs to be long enough for the effects of the stress to be fully evident. For some risks, this may require stress testing over a complete economic cycle.”
“in adverse situations, previously low levels of correlation can increase. Determining interdependency requires judgment, as there may be no historical data that throw meaningful light on new social and economic conditions.”
Reverse stress testing – firms should identify scenarios and stresses that could result in the failure of the firm. This should help to identify a firm’s core vulnerabilities.
“while some risk of failure is always present, reverse stress testing may help to ensure adequate focus on the management actions that are appropriate to avoid undue risk of business failure.”
Management actions – firms should consider whether to allow management actions12 in response to stresses.
Documentation – firms should document the governance, structures, policies and procedures, and results of stress testing.
“policies and procedures should cover all aspects of the stress testing framework, be clearly documented, kept up to date, and be approved by the board and/or senior management.”
“a supervisor should receive the results of the most material stress tests and the critical assumptions underlying them, and have access to full details on the assumptions and methodology used by the firm in its stress testing.”
Review and challenge – there should be an independent consideration of scenarios, stresses, assumptions, and models, including by internal audit.
“a level of independence should exist to ensure that an adequate set of tests has been designed that is appropriate to the risk profile of the firm. Decisions about the tests undertaken should be made, if possible, by those who are not involved in the corresponding business decisions.”
“the stress testing framework should facilitate credible challenge of the stress testing framework, both at senior and technical expert levels, including not only assumptions, methodologies, scenarios and results, but also the assessment of its ongoing performance and effectiveness, and the remediation of gaps identified by key stakeholders.”
“the scenarios and sensitivities used in stress tests should be reviewed periodically to ensure that they remain relevant. Consideration should be given to historical events and hypothetical future events that take into account new information and emerging risks in the present and foreseeable future.”
“as with any critical management process, the independent audit function should regularly review the firm’s stress testing framework and its implementation.”
Usage – stress testing should be a key input into a firm’s risk identification, monitoring and assessment activities; and into the formulation of strategic objectives.
“stress tests should be a fundamental element of a firm’s overall risk management framework and capital and liquidity adequacy determination. They should help the firm in making decisions as to whether, and what, action is needed to ensure that it is not taking undue risks.”
“when using the results of stress tests, firms should have a clear understanding of their key assumptions and limitations, for instance in terms of scenario relevance, risk coverage and model risk.”
“the ICAAP or ORSA should incorporate stress testing to complement and help validate other quantitative and qualitative approaches so that the firm’s management may have a more complete understanding of the firm’s risks and the interaction of those risks under stressed conditions.”
National supervisory authorities need to decide what rules, guidance and other supervisory expectations to issue on stress testing by the financial institutions they supervise. These could be stand-alone rules and guidance relating specifically to stress testing13, or they could be part of broader rules and guidance on risk management14 and on the ICAAP/ORSA process.15 These rules and guidance should be based on the international standards listed above, including the principle of proportionality.
It will be easier for supervisors to assess a financial institution’s stress testing and to intervene accordingly if there are at least some rules or guidance issued by the supervisory authority on stress testing by financial institutions.
Supervisory authorities should also consider the benefits of rules and guidance relating to new areas, for example financial institutions using scenario analysis and stress testing to inform their understanding of the potential impacts of cyber security risks and of climate and biodiversity-related risks.
What can go wrong in stress testing by financial institutions?
Basel Committee (2009) reviewed a range of weaknesses in stress testing practices employed by banks ahead of the Global Financial Crisis. Although bank-focused, these weaknesses are also applicable to other sectors.
These weaknesses illustrate why supervisors need to conduct their own assessment of the quality of a financial institution’s stress testing and point to some specific areas on which supervisors should focus.
Insufficient engagement of boards and senior management - boards and senior management were not sufficiently engaged in defining scenarios and stress tests, discussing the results of stress tests, challenging models and assumptions, and acting upon the results for capital and liquidity planning.
This lack of board and senior management engagement also meant that there was an insufficient integration of stress testing with a firm’s strategy, business model, risk appetite, risk management and recovery planning.
Weak organisational structure for stress testing – in some cases, stress testing was performed mainly as an isolated exercise by the risk function or the finance department, with little interaction with business areas. Alternatively, stress tests were undertaken by separate business areas focusing on specific business lines or risk types, with only limited firm-wide integration.
As a result, there was insufficient identification and aggregation of risks on a firm-wide basis; a lack of a comprehensive firm-wide perspective across risks and business lines; and an inability to identify correlated tail exposures and risk concentrations across the firm.
Mechanical approach – some firms relied too heavily on routinely repeated stress tests. While there may be a place for these within a comprehensive stress testing programme, they cannot take account of changing business activities or changing conditions.
Inadequate processes, data and IT to run effective stress tests – many firms lacked the “basics” needed to run stress tests, including clear internal processes, sufficiently granular and available risk information, the ability to aggregate risk exposures across the firm, and IT infrastructure.
Poor scenario and stress test design – some firms were using overly optimistic scenarios and imposing insufficiently severe stresses, including an under-estimation of the potential severity and duration16 of stress events.
There was also a failure to stress all key vulnerabilities, and inadequate coverage of risks arising from complex structured products, securitization risk, counterparty credit risk, risks arising from credit lines, reputational concerns related to off-balance sheet vehicles, and liquidity and funding risks arising from the systemic nature of the crisis.
Inadequate attention to feedback loops and market-wide stresses - inadequate account was taken of system-wide interactions and feedback effects caused by market reactions to stressed conditions. For example, stress tests failed to capture the extent to which known and unknown losses on structured products led to a sustained market-wide disruption to interbank markets.
Inadequate modelling techniques – most risk management models, including stress tests, used historical statistical relationships to assess risk. They assumed that risk is driven by a known and constant statistical process, so that future risks can be predicted from historical relationships.
However, following a long period of stability, backward-looking historical information did not reflect the possibility of severe shocks or the build-up of vulnerabilities within the system. The severity levels and duration of stress indicated by previous episodes proved to be inadequate; historical data could not capture all the risks in new products that were at the centre of the crisis; and historical statistical relationships, such as correlations, proved to be unreliable once actual events started to unfold.
Moreover, risk characteristics can change rapidly in stressed conditions as reactions by market participants within the system can induce feedback effects and lead to system-wide interactions that can dramatically amplify the initial shocks.
The management of most firms did not question sufficiently these limitations of more traditional risk management models, and did not take sufficient account of qualitative expert judgment to develop innovative ad-hoc stress scenarios.
Meanwhile, even where firms used hypothetical stress tests to capture events that had not yet been experienced, these tests were generally only moderate in terms of their severity, duration and the degree of interaction and correlation across portfolios or risk types. Scenarios that were considered extreme or innovative were often regarded as implausible by the board and senior management. Even supposedly “severe” stress scenarios typically resulted in estimates of losses that were no more than a quarter’s worth of earnings (and typically much less).
Over-optimistic reliance on management actions – many firms made over-optimistic assumptions about the extent and speed at which they would be able to adjust as shocks occurred, particularly during market-wide stresses.
Some of the lessons from the Global Financial Crisis were repeated in Silicon Valley Bank (SVB) ahead of its failure in March 2023.17
First, even on a simple “single factor” sensitivity test basis, the bank’s own stresses of deposit withdrawal were insufficiently severe, given that a large proportion of the bank’s deposits were:
- deposited by a relatively small and concentrated depositor base of technology and life science companies that were reliant on investments from venture capitalists;
- mostly non-interest-bearing on-demand deposits, so vulnerable to increasing interest rates;
- nearly all (up to 95 per cent) above the $250,000 coverage limit of the deposit insurance scheme (the FDIC); and
- easily accessible and movable 24 hours a day, every day, through mobile phone applications and internet-based accounts.
Social media enabled depositors to instantly spread concerns about the bank, and technology enabled an immediate withdrawal of deposits.
Regulatory requirements such as the Liquidity Coverage Ratio (LCR) – which is itself in effect a stress test – would also have been inadequate because SVB suffered considerably more rapid deposit withdrawals than are assumed under the LCR.
Indeed, as shown by the experience of Credit Suisse in March 2023, serious funding problems can threaten even a global systemically important financial institution subject to the full panoply of Basel III liquidity ratios, additional capital requirements, higher expected standards of governance and risk management, more intensive and intrusive supervision, and recovery and resolution planning.18
Firms (from all sectors) should therefore use liquidity stress testing as a forward-looking risk management tool to reveal vulnerabilities in the firm’s liquidity profile and provide information on its ability to meet liabilities as they fall due. This should be based on:
- sufficiently severe (but plausible) market-wide and firm-specific scenarios and stresses that include hypothetical (“black swan”) scenarios;
- careful consideration of the potential impact of stresses on the availability and cost of all external and intra-group sources of funding, and on funding commitments given and received by the firm;
- careful consideration of the terms on which holdings of assets can be sold, or borrowed against, to meet funding needs; and
- second-round and systemic effects.
Second, over-optimism. SVB repeatedly failed its own internal liquidity stress tests, but in response to this the management of the bank switched to using less conservative stress testing assumptions, which masked some of the risks; assumed that the highly concentrated deposit base was more stable than it proved to be; and made unrealistic assumptions about available funding resources in a stress scenario.
Third, insufficient attention was paid to interactions between liquidity and capital. Market concerns about the realised and unrealised losses on the bank’s holdings of fixed income securities led to deposit withdrawals on a scale that – in the absence of alternative sources of funding – would have required the bank to begin selling its portfolio of “held to maturity” securities, which would have triggered the realisation of the losses on this portfolio and would have almost wiped out the bank’s capital.
A stress scenario based around rising interest rates would have identified the impact of increases in interest rates on both the value of fixed income securities and the likely withdrawal of non-interest-bearing deposits, and the self-reinforcing interactions between these two impacts. An increase in interest rates was an entirely predictable scenario - market participants and analysts in the US had been warning for some time of the possibility of much higher interest rates as inflationary pressures (and inflation itself) increased significantly.
Stress testing by financial institutions is important for supervisors for two main reasons.
First, it should provide information to supervisors about the potential consequences of the main risks facing a financial institution, and the financial resources that might be required to absorb losses (or to provide funding) should large shocks occur.
This can help supervisors in assessing:
- The inherent risks in a financial institution’s business activities.
- The vulnerabilities generated by a financial institution’s business model.19
- The capital, solvency and liquidity required to support a financial institution.
- The quality of a financial institution’s capital, solvency and liquidity planning20, and more specifically the quality of its ICAAP or ORSA.
- The quality of a financial institution’s recovery plan.21
Second, the way in which a financial institution runs, governs and uses its stress tests can inform supervisors about the quality of a financial institution’s governance and risk management.
These two types of supervisory assessment can in turn feed into many elements of a risk assessment matrix under a risk-based approach to supervision22, or into the assessment of the risk management, capital and liquidity elements within a CAMELS type approach.
This section discusses four areas that supervisors should focus on in their assessment of a financial institution’s scenario analysis and stress testing, and suggests some lines of questioning that supervisors could use to assess each area. The four areas are:
- High level approach.
- Governance
- Technical competence.
- Use of the results.
In assessing these areas supervisors should – as with other types of supervisory assessment – consider the balance between off-site and on-site supervision, and when it might be helpful to utilise expert resources.
Off-site and on-site supervision
Off-site supervision refers here to the supervisory review and evaluation of documentation from financial institutions. Some of the documents relevant to stress testing will be received as part of general reporting requirements (for example an ICAAP or ORSA submitted by a bank or an insurer), or as part of the extensive documentation received from financial institutions on their governance and risk management.
In addition, supervisors can request documentation that details the scenario analysis and stress testing undertaken by a financial institution; the data, models and assumptions used for this analysis and testing; and the internal procedures for how this analysis and testing is reported and discussed within the financial institution.
For larger and more significant financial institutions this information can be supplemented with on-site discussions with directors, senior management and relevant staff. These discussions will include both (a) largely factual questions to clarify or extend the information from the documentation received and reviewed off-site; and (b) more “open ended” questions designed to help supervisors to form an assessment of the quality and effectiveness of a financial institution’s scenario analysis and stress testing, how the relevant processes and procedures operate in practice, and how the results are discussed and used as the basis for decision-making.23
Supervisors can make considerable progress in an assessment of a financial institution’s scenario analysis and stress testing without a high level of expert knowledge. However, the assessment of more technical aspects might require more expert resources. As in other technical areas (for example, risk modelling, IT systems and cyber security) these expert resources could be from a specialist team within the supervisory authority; from other authorities such as a central bank (or from another department of the central bank where the central bank is also a supervisory authority); or from external firms that could be hired by either the supervisory authority or by a financial institution (at the request of the supervisory authority) to undertake a third-party review.
A financial institution’s high level approach to stress testing
Supervisors might usefully begin by considering:
- the scenario analysis and stress tests (if any) that a financial institution currently undertakes;
- how frequently these are undertaken; and
- whether the financial institution has plans to develop further its scenario analysis and stress testing.
Rules and guidance issued by a supervisory authority – or internal guidance to supervisors - might specify the main types of scenario analysis and stress testing that a financial institution in a particular sector is required or expected to undertake, and how a proportionate approach could be taken (for example, by differentiating financial institutions according to their systemic importance, size, complexity, or activities).
In addition, a supervisory authority might have announced an expectation that at least some financial institutions should be developing their scenario analysis and stress testing of climate-related or cyber security risks; or that in the light of the failures of SVB and Credit Suisse banks should be undertaking more severe stress tests of deposit withdrawals.
The table below provides an example of the questions that supervisors might address, through both off-site and on-site supervision. These questions (in each of tables 1-4) are intended to be illustrative rather than comprehensive, recognising that (a) they may need to be fine-tuned to correspond more closely to the rules and guidance issued by a supervisory authority; (b) they cannot cover all aspects of the scenario analyses and stress tests run by every financial institution; and (c) the answers are likely to give rise to follow-up lines of questioning.
It should be reasonably self-evident which questions should be directed to which individuals within a financial institution, including members of the board, senior management, heads of business units, and heads of control functions. Some of the questions might usefully be asked of more than one individual, not least to see whether multiple responses reveal consistent or inconsistent answers.
Table 1: High level approach
|
Topics |
More detailed questions |
|
What scenarios and stress tests does a financial institution (FI) use? To what extent do these cover macroeconomic, product-specific, and climate and biodiversity scenarios and stresses? |
Why and how has the FI chosen these scenarios and stresses? What are the objectives of these scenarios and stresses? How does the FI justify its choice of scenarios? Does the FI recognise that the scenarios it is using may not be sufficiently wide-ranging? What macroeconomic variables (individually or in combination) might be most relevant to the risks run by the FI? Have these been included in the FI’s scenario analysis and stress testing? Has the FI considered the use of any climate risk and biodiversity loss scenarios and stress tests? |
|
Do the scenarios and stress tests cover the main risks run by the FI?
Are they reasonably comprehensive? |
Can the FI explain how its stress tests relate to the material risks run by the FI? For example, where the FI is a bank, how do its stress tests capture its credit and liquidity risks? Where the FI is an insurer, how do its stress test capture its underwriting risks, and the market and credit risks on the asset side of its balance sheet? Where the FI is a position-taking securities firm, how do its stress tests capture its market and funding risks? Is the FI concerned that some material risks may be missed by its stress tests? Has the FI analysed how comprehensive its stress tests are in comparison with the main risks it is running (for example its credit, insurance, market, liquidity and operational risks)? Has the FI considered more granular stress testing for sub-categories of its main risks (for example of different types of credit or insurance underwriting risk)? What stress tests has the FI considered using, but not pursued? Why were they not pursued? |
|
How severe and plausible are the scenarios and stress tests used by the FI? |
How did the FI decide on the severity of its scenarios and stress tests? How would the FI justify these choices? How did the FI decide on the duration of its scenarios and stress tests? Is it making an over-optimistic assumption that any stresses will be short-lived? Did the FI consider running scenarios and stress tests that apply more severe stresses, including reverse stress tests? What value does the FI see in reverse stress testing? |
|
Do the stress tests include second-round and feedback effects (for example interactions between liquidity and solvency, between the funding of the FI and market-wide funding stresses, between catastrophes and asset values, or between asset sales and the price of those assets)? |
How did the FI decide on which interactions to include in its stress testing, and how to do this? Which interactions does the FI consider to be the most important? How has the FI captured the possibility that solvency problems could create funding problems, and that liquidity problems could create solvency problems? How has the FI captured direct and indirect contagion effects, and other types of interconnectedness? How has the FI captured market-wide stresses? |
|
How frequently does the FI run its stress tests? |
How did the FI decide on the frequency of its stress tests? How often are scenarios and stress tests reviewed and updated? What examples can you provide of where this updating has occurred? Have stress tests been run more frequently, or revised, in response to macroeconomic, market, climate, cyber attacks, and other developments? |
|
Where the FI is part of a wider group, is stress testing undertaken on a solo or group-wide basis (or both)? |
How does the FI decide on whether to run solo or group-wide stress tests? What does the FI see as the advantages and disadvantages of solo and group-wide stress tests? How does the FI test:
|
A financial institution’s governance arrangements, policies, processes and procedures for scenario analysis and stress testing should be documented, and therefore capable of being made available for off-site review by a supervisor.
This should include, for example:
- Roles and responsibilities of the board, senior management, and a range of a FI’s staff, as they relate to scenario analysis and stress testing.
- Organisational structure.
- Policies and procedures for scenario analysis and stress testing.
- The ways in which all three lines of defence (business units, risk management, and internal audit) are involved in scenario analysis and stress testing.
- The available resources and expertise for scenario analysis and stress testing, including the use of third-party expertise and input.
However, the documentation may provide only a partial, or even a misleading, view of what happens “on the ground.” It can be supplemented by on-site questioning (see Table 2).
Table 2: Governance and processes
|
Topics |
More detailed questions |
|
Documentation |
Is the documentation complete? What is not documented, or not documented adequately? Why have any gaps in the documentation of scenario analysis and stress testing arisen? How structured is the scenario analysis and stress testing process? |
|
Roles and responsibilities
|
Are roles and responsibilities relevant to scenario analysis and stress testing clearly specified and documented? Ask a range of individuals at different levels within the FI what their roles and responsibilities are with respect to scenario analysis and stress testing. Are their answers consistent with the documentation, and with each other? |
|
Role of the board |
When did the board last discuss the selection, design and assumptions underlying the scenario analysis and stress tests that the FI should be undertaking? What happened at that board discussion? What, if anything, changed as a result of the discussion? If the FI has a Board Risk Committee, what role does it play in scenario analysis and stress testing? When did the Risk Committee last discuss these issues? What was the nature of the discussion? What was reported and recommended to the board as a result? Using a recent stress test undertaken by the FI as an example, what contributions were made by the board and senior management? Who provided challenge, and what changes did this lead to? |
|
Organisational structure
|
Is the organisational structure for scenario analysis and stress testing clear? Are different scenario analyses and stress tests undertaken by different parts of the FI? If so, how are these coordinated and brought together to provide a firm-wide view? If scenario analysis and stress tests are undertaken by a central function, how do other areas of the FI contribute to this? |
|
Policies and procedures
|
Which policies and procedures relating to scenario analysis and stress testing are documented? Ask a range of staff about their knowledge of the policies and procedures that are relevant to their roles in scenario analysis and stress testing. Are their answers consistent with the documented policies and procedures (where they exist), and with each other? Have there been cases where policies and procedures have been by-passed? Why did this happen? |
|
Three lines of defence
|
What are the roles and responsibilities of the three lines of defence in relation to the FI’s scenario analysis and stress testing? Ask a range of staff about their knowledge of the roles and responsibilities of the three lines of defence (business units, risk management, and internal audit) in the FI’s scenario analysis and stress testing. Are their answers consistent with the documented policies and procedures (where they exist), and with each other? How does the FI safeguard against the front-line business units taking, or imposing, an over-optimistic view of likely stresses, or of their likely impact? Does the risk management function run scenario analyses and stress tests? If so, how can it provide independent challenge? What has internal audit reviewed in practice with respect to any aspect of scenario analysis and stress testing? What did it find? Have its recommendations been acted upon? |
|
Resources and expertise |
Where an FI uses its own staff to undertake scenario analysis and stress testing, how has the FI determined what experience and expertise they need? How does the FI ensure that these staff have the necessary experience and expertise? What training relating to scenario analysis and stress testing does the FI provide to its board and senior management? Where an FI uses a third party for any aspect of its scenario analysis and stress testing, how does the FI assure itself that the third party has the necessary experience and expertise? How does the FI ensure that information and learning are not “lost” as a result of outsourcing? |
A financial institution should have an adequate (and proportionate) technical approach to scenario analysis and stress testing.
This can become quite a daunting technical area for supervisors, especially where they are supervising larger and more sophisticated financial institutions. Supervisors may therefore lack confidence in exploring and assessing the technical approaches being taken by such financial institutions. Supervisors should therefore consider calling upon experts in this area, as discussed above, to help with this assessment.
Most of the technical issues here are the same as those applying to a supervisory authority or central bank running its own scenario analysis and stress testing. Toronto Centre (2023b and 2024b) provides useful background information on some of the more technical issues.
The questions listed in Table 3 are high level and do not delve into the finer details of modelling methodologies. Supervisors ought to be aware of how a financial institution approaches these issues, even if they rely on expert resources to review the finer details.
Model specifications can be reviewed off-site, but it should also be beneficial for supervisors (together with technical experts) to take advantage of on-site visits to discuss with major financial institutions why they taken their chosen approach(es) and what issues have arisen.
Table 3: Technical approach
|
Topics |
More detailed questions |
|
Approaches taken by the FI to scenario analysis and stress testing |
Why has the FI adopted its specific approaches to scenario analysis and stress testing? Where it is using sensitivity testing, how are these stress tests quantified? How does the FI take account of the broader picture when using sensitivity testing? Where an FI is using one or more scenarios, how does the FI model the “transmission mechanism” between each scenario and its impact on the FI? How does the FI choose between modelling using historic data and hypothetical modelling based on expert assumptions or Monte Carlo type simulations? How does the FI assess whether this modelling of the “transmission mechanism” generates plausible outcomes, for example on losses, earnings, solvency and other balance sheet items? Does the FI run “exploratory” stress tests for newer risks such as climate and biodiversity-related risks? How does the FI decide whether and when to run such stress tests? See also Box 1. |
|
Data |
Does the FI have adequate data to support its approach to scenario analysis and stress testing? What data does the FI think it needs? Where does the FI find these data? How does the FI assess the quality of these data? What actions has the FI taken to improve these data? |
|
IT systems
|
Does the FI have adequate IT systems to support its approach to scenario analysis and stress testing? What IT capability does the FI think it needs? Where does the FI find this capability? What limitations remain? What actions has the FI taken to improve its IT systems? Does the FI make use of outsourced IT systems to support its stress testing? If so, how does it assess the quality of the services provided by third parties? |
|
Assumptions and use of expert judgement
|
Where, and to what extent, do the scenario analyses and stress testing undertaken by the FI rely on assumptions and expert judgement? How are these assumptions and expert judgements generated? How does the FI assess the quality of these assumptions and expert judgements? Do the board and senior management understand and discuss these assumptions and expert judgements? |
|
Modelling
|
What models does the FI use to support its scenario analysis and stress testing? How and why has the FI chosen to use these models? Do these models cover a sufficient range of outcomes, given the degree of model uncertainty and the key tail risks? Which models does the FI have least confidence in? Why? How does the FI assess whether historical relationships can still be relied on? How does the FI decide what time horizons to use for its scenarios and stress tests? What difference does this choice make in practice? |
|
Management actions
|
Does the FI allow for management actions in its stress testing? Where it does, how does the FI ensure that this does not lead to over-optimistic or implausible outcomes? What processes and challenges does the FI have in place to assess the reasonableness of any assumed management actions? Are any assumed management actions subject to sign-off by the board? |
|
Model validation and challenge |
What model validation and challenge procedures does the FI have in place for its scenario analyses and stress testing? How have these procedures changed the scenarios, stress tests, models and data used by the FI? |
|
Review and updating |
How often does the FI review and update its scenarios and stress tests? Who is involved in this review and updating? What changes has the FI made as a result of this? |
Box 1: Climate and biodiversity-related risks
|
Major financial institutions should be expected to undertake scenario analysis and stress testing of the climate and biodiversity-related risks they face. As discussed in Toronto Centre (2023a and 2023b), this raises some analytical and modelling difficulties, not least in developing scenarios and in translating these into the potential impact on the financial institution. Where financial institutions are attempting to assess the impact of them on climate and biodiversity-related risks, supervisors should ask them: Scenarios What scenarios is the financial institution using, and how were these selected? Are these scenarios selected from the range of scenarios specified by international organisations such as the Network for Greening the Financial System (NGFS), the International Actuarial Association, and the Intergovernmental Panel on Climate Change? Is the financial institution using variants of these international scenarios to reflect national circumstances more accurately? Are these scenarios sufficiently severe and plausible, taking into account the specific circumstances of the financial institution, including its vulnerability to physical and transition risks? Where a financial institution may be vulnerable to biodiversity risks, what scenarios or “narratives” is it using?24 Transmission channels How is the financial institution translating its chosen scenarios into impacts on (i) broad economic variables such as GDP, employment and inflation; (ii) regions and sectors of the economy; and (iii) specific physical and transition risks relevant to the location and the types of business undertaken by the financial institution? What models or other approaches is the financial institution using here? Why has the financial institution chosen these models and approaches? How is the financial institution taking account of:
|
Financial institutions should make use of the results of their scenario analysis and stress testing to improve their identification and understanding of risks; monitor and assess their business activities; formulate their strategic objectives and risk appetite; improve their decision-making; and assess the adequacy of their capital and liquidity.
Supervisors can use off-site and on-site supervision to assess and evaluate how financial institutions use the results of their scenario analysis and stress testing. The main supervisory concerns in this respect are usually that (a) the results are not properly reported and understood within a financial institution; and (b) the financial institution does not use the results effectively.
Table 4: Using the results
|
Topics |
More detailed questions |
|
Upward reporting
|
To whom and how are the results of stress tests reported? What reports are sent to the board and to senior management? Do these reports cover all stress tests run by the FI? If not, what determines the content of these reports? Is this a clearly understood and documented process? What “interventions” are typically made during this upward reporting? Have there been any instances where the results of stress tests have been adjusted, or where different stress tests have been run before the results were reported upwards? |
|
Board and senior management discussion of the results
|
What do senior management and the board do with these reports? What discussions typically take place? When did the board last discuss the results of scenario analysis and stress tests undertaken by the FI? What happened at that board discussion? What examples are there of how stress testing has helped senior management and the board to understand better the risks being taken by the FI? |
|
Input into decision making
|
What actions and outcomes have stress testing led to? How have the results of stress testing influenced the FI’s: • Strategy • Risk appetite • Limits and controls • Business decisions • Capital planning and liquidity planning • Assessment of the adequacy of its capital and other financial resources, and its ICAAP or ORSA • Recovery planning? What examples can the FI provide on each of these? |
|
Learning lessons
|
What has the FI learnt from its scenario analysis and stress testing? How has this changed its approach to scenario analysis and stress testing? |
Supervisory response and intervention
Supervisors should review and evaluate a financial institution’s scenario analysis and stress testing, to assess these against any rules, guidance and supervisory expectations issued by the supervisory authority, and against the four areas of assessment discussed above.
As a result of this assessment supervisors could take a range of actions (if they have the relevant powers to do so), including requiring a financial institution to:
- Improve its governance and risk management, where its approach to scenario analysis and stress testing has revealed weaknesses.
- Use a wider range of scenarios and stresses, to capture a wider range of the risks facing a financial institution.
“where the supervisory authority considers that the stress tests conducted by the firm should be supplemented with additional tests, they should be able to require the firm to carry out such additional tests.” - Use more challenging scenarios and to run more severe stress tests, to capture more severe but still plausible scenarios and stresses.
- Improve the technical aspects of its scenario analysis and stress testing, for example with respect to data and modelling, any assumptions used, a reliance on management actions, and the time horizon of stresses.
- Hold additional (“Pillar 2”) capital or liquidity, or amend its strategy and business activities, to protect the financial institution against the possibility that a scenario or stress might materialise.
“where the supervisor feels that the firm’s response to the results of the stress test is insufficient, it should be able to direct the firm to develop a more prudent response.”
“if necessary, the supervisor may require the firm to increase its capital, strengthen its systems and controls, or amend its business plan and strategies.” - Run one or more “top-down” stress tests at the request of the supervisory authority. This would usually be a common stress test for all financial institutions of a particular type/size/importance (as discussed in Toronto Centre (2024b)); but it could also be an institution-specific requirement, reflecting institution-specific risks as identified by the supervisor.
“there are circumstances where the supervisor may develop standard stress tests and require firms to perform such tests. Such tests may be directed at a single firm, selected firms, or all firms.”
In addition, a supervisory authority should consider: - Issuing rules and guidance on scenario analysis and stress testing by financial institutions, to address common failings across financial institutions and new sources of risk such as cyber security and climate and biodiversity-related risks.
“It would be appropriate for the supervisor to establish the requirements for stress testing for prudent risk management purposes for the firm, for example the nature and minimum frequency of such tests. Some jurisdictions, in addition to requiring stress testing, may also prescribe broad minima for the factors that the testing must address.” - Drawing out and publishing examples of good and less good practice, as observed from the assessment of scenario analysis and stress testing practices across financial institutions. The examples of good practice might also be an input to the development or extension of rules and guidance in this area.
- How the risk-based (or other) approach to supervision used by the supervisory authority should be adjusted to incorporate the results of scenario analysis and stress testing into the risk assessment of a financial institution, including its inherent risks, its governance and controls, and the adequacy of its financial resources.
This Note has outlined the expectations that supervisory authorities might have, on a suitably proportionate basis, for the scenario analysis and stress testing undertaken by financial institutions as part of their own risk management (in addition to any “top-down” stress testing imposed on financial institutions by a supervisory authority or a central bank).
The Note has also discussed how supervisors can assess and evaluate the quality and effectiveness of a financial institution’s scenario analysis and stress testing, including through the on-site questioning of board members, senior management and other staff in a financial institution.
This supervision might usefully focus on whether a financial institution’s scenario analysis and stress testing is sensible, and proportionate to the size, nature and complexity of the financial institution; well governed; technically competent; and used effectively by the financial institution.
Finally, supervisors can make a range of interventions, depending on the powers available to them, to impose various requirements on a financial institution, and to take sector-wide actions such as issuing rules and guidance.
Basel Committee on Banking Supervision. Principles for sound stress testing practices and supervision. May 2009.
Basel Committee on Banking Supervision. Core Principles for Effective Banking Supervision. September 2012.
Basel Committee on Banking Supervision. Stress testing principles. October 2018.
Basel Committee on Banking Supervision. Supervisory review process. SRP30. Risk management. December 2019.
European Banking Authority. Guidelines on institutions’ stress testing. July 2018.
Federal Reserve Board. Review of the Federal Reserve’s Supervision and Regulation of Silicon Valley Bank. April 2023
Financial Stability Institute. Supervisory practices for assessing the sustainability of banks’ business models. April 2022.
FINMA. Lessons Learned from the Credit Suisse Crisis. December 2023.
Hong Kong Monetary Authority. Supervisory Policy Manual: Stress testing. May 2021.
International Association of Insurance Supervisors. Stress testing by Insurers Guidance paper. October 2003.
International Association of Insurance Supervisors. Insurance Core Principles. November 2019.
Monetary Authority of Singapore. Guidelines on risk management practices – Credit risk. March 2013.
Monetary Authority of Singapore. Guidance on insurers’ own risk and solvency assessments. July 2017.
Network for Greening the Financial System. Recommendations toward the development of scenarios for assessing nature-related economic and financial risks. December 2023.
Toronto Centre. Risk-based supervision. March 2018.
Toronto Centre. Stress testing and climate change. June 2020a.
Toronto Centre. Pillar 2 and beyond: Issues for supervisors in implementing Basel II and Basel III. June 2020b.
Toronto Centre. Recovery planning. August 2020c.
Toronto Centre. Operational resilience: the next frontier for supervisors. April 2021.
Toronto Centre. Supervising Corporate Governance: Pushing the Boundaries. April 2022.
Toronto Centre. A Climate and Biodiversity Risks Toolkit for Financial Supervisors. February 2023a.
Toronto Centre. Introduction for supervisors to scenarios and stress tests of climate change risks. May 2023b.
Toronto Centre. Cyber Risk: Determining and delivering a supervisory strategy. July 2023c.
Toronto Centre. The Risk-Based Supervision of Liquidity. February 2024a.
Toronto Centre. Supervisory Stress Testing: A Primer. February 2024b.
1This Toronto Centre Note was prepared by Clive Briault. Please address any questions about this Note to This email address is being protected from spambots. You need JavaScript enabled to view it.
2See Toronto Centre (2020a, 2023b and 2024b).
3This Note is mostly about financial resilience. Toronto Centre (2021) covers operational resilience, while Toronto Centre (2023c) focuses on cyber risk.
4This own assessment should cover liquidity as well as capital, either within an ICAAP or ORSA, or in a separate ILAAP. See Toronto Centre (2020b and 2024a).
5Recovery planning is discussed in Toronto Centre (2020c).
6Toronto Centre 2020a, 2023b and 2024b.
7Hypothetical scenarios could be based on shocks that have occurred in other countries.
8Various modelling techniques can be used here, including deterministic modelling and various types of stochastic modelling, including Monte-Carlo simulation approaches. See Toronto Centre (2024b).
9These risks are explained in Toronto Centre (2023a).
10The quotations used here, and throughout the rest of this Note, are taken from Basel Committee (2018 and 2019), and IAIS (2003 and 2019). The quotations are of cross-sector relevance, so “bank” and “insurer” have been replaced by “firm”.
11See in particular Basel Core Principle 15 on Risk Management Process (Basel Committee (2012)), and Insurance Core Principle 16 on Enterprise Risk Management for Solvency Purposes (IAIS (2019)).
12Actions that could be taken by a financial institution in response to a stress, to moderate its impact.
13See, for example, European Banking Authority (2018), and Hong Kong Monetary Authority (2012).
14See, for example, Monetary Authority of Singapore (2013).
15See, for example, Monetary Authority of Singapore (2017).
16An important characteristic of stress tests is that their impacts depend on their assumed duration, as well as on the magnitude of the “shock” and the assumptions made in specifying the transmission mechanisms through which the shock feeds through to a financial institution.
17The events leading up to the failure of SVB are described in Federal Reserve Board (2023).
18See FINMA (2023).
19See Financial Stability Institute (2022) on business model analysis and the use of scenario analysis and stress testing in that context.
20Toronto Centre (2024a) discusses the role of scenario analysis and stress testing in the assessment of liquidity as a financial resource within risk-based supervision.
21See Toronto Centre (2020c) for a discussion of recovery planning.
22Toronto Centre (2018).
23The use of open-ended questions in related areas of supervisory assessment is discussed in Toronto Centre (2022) on corporate governance and Toronto Centre (2024a) on liquidity.
24Network for Greening the Financial System (2023) discusses the development of biodiversity loss scenarios.