discussion on “diagnosing the financial system: financial conditions and financial stress” by...
TRANSCRIPT
![Page 1: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/1.jpg)
Discussion on “Diagnosing the Financial System: Financial Conditions
and Financial Stress”
by Scott Brave and R. Andrew Butters
Discussant: Chen ZhouDe Nederlandsche Bank
Views expressed do not necessarily reflect the view of DNB
![Page 2: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/2.jpg)
The paper
• An indicator of financial condition (NFCI)• Thermometer for financial system• Based on a large scope of macro/finance data• Aggregating data with different frequency
• NFCI as an indicator/predictor for financial stress• Normal v.s. stress (crisis threshold)• ROC curve in threshold selection
![Page 3: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/3.jpg)
An example in diagnosis
• Diagnosing a rare disease• Very rare disease Pr(Virus)=0.01%• Very reliable doctor/method
• P(Positive|Virus)=99.9%• P(Negatie|No Virus)=99.9%
• When getting a positive test result, should the patient worry?
Calculating Pr(Virus|Positive)
![Page 4: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/4.jpg)
Calculation…
• Bayes formula
The patient does not have to worry too much!
%0.99.9999.99
9.99%99.99*%1.0%01.0*%9.99
%01.0*%9.99
)Pr()|Pr()Pr()|Pr(
)Pr()|Pr(
)|Pr(
NoVirusNoVirusPositiveVirusVirusPositive
VirusVirusPositive
PositiveVirus
![Page 5: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/5.jpg)
What went wrong with the diagnosis?
• Why Pr(Virus|Positive) is not high• The diagnosis method seems to be reliable:
Pr(Positive|Virus) and Pr(Negative|No Virus) are both very high
• However, Pr(Virus|Positive)≠Pr(Positive|Virus)• If Pr(Positive)>P(Virus)
Pr(Virus|Positive)<Pr(Positive|Virus)• If a positive signal occurs too often, then it is
less informative.
![Page 6: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/6.jpg)
A Close look at the formula
• As Pr(Virus) tends to zero, Pr(Virus|Positive) can be very low. Hence it is very difficult to diagnose rare disease.
)Pr()|Pr()Pr()|Pr(
)Pr()|Pr(
)|Pr(
NoVirusNoVirusPositiveVirusVirusPositive
VirusVirusPositive
PositiveVirus
)Pr(/)Pr()|Pr(/)|Pr(
)|Pr(/)|Pr(
)|Pr(
VirusNoVirusNoVirusPositiveVirusPositive
NoVirusPositiveVirusPositive
PositiveVirus
Very high
Should be very high to offset the effect that the disease is rare
999 9999
![Page 7: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/7.jpg)
Lessons learned
• Diagnosing rare disease• Having low Type I and II errors is not sufficient• It is necessary to have a diagnosis system that
do no produce many “positive” signals.• It is necessary to have Type II error much
lower than Type I error
Rare disease is very difficult to diagnose by nature!
![Page 8: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/8.jpg)
Back to diagnosing the financial system
• How about the NFCI as an indicator?• Financial stresses/crises are rare:
• In this paper P(Crisis)=52/1983=2.62%• With the threshold at -0.37, from ROC
• Pr(Signal|Crisis)=91%• Pr(Signal|No Crisis)=19%
• Thus Pr(Crisis|Signal)=11.4%, quite low!• Reason: Pr(Signal)=20.9%>>Pr(Crisis)
• See also Figure 3
![Page 9: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/9.jpg)
How to improve?
• To diagnose rare events, it is necessary to have Type II error much lower than Type I error
• Currently: Type II error: 19%, Type I error 9%• Along the ROC curve, can we improve?
Here?
Type II: 2%
Type I: 38%
![Page 10: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/10.jpg)
At the new point
• Calculations:• P(Crisis)=52/1983=2.62%• Pr(Signal|Crisis)=62%• Pr(Signal|No Crisis)=2%• Thus Pr(Crisis|Signal)=45.5%• Better, but still not satisfactory
• Threshold:• Much higher than -0.37 at the new point!
![Page 11: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/11.jpg)
Which conditional probability?
• Comparison• In current paper:
Pr(Signal|Crisis) and Pr(No Signal|No Crisis)• We talked about:
Pr(Crisis|Signal) and Pr(No Crisis|No Signal)• Why the latter is more important than the
former?• Utility function: decisions from the signal
![Page 12: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/12.jpg)
Can we improve?
• Absolute performance• Difficult to have Pr(Signal|Crisis) and
Pr(No Signal|No Crisis) both at high level• Not only NFCI, but also other indicators• Difficult as it is
• Relative performance• Can still be compared• Utility function based on the new sets of
conditional probability
![Page 13: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/13.jpg)
An alternative ROC curve• Current ROC curve based on
Pr(Signal|No Crisis) v.s. Pr(Signal|Crisis)• New ROC curve based on
Pr(Crisis|No Signal) v.s. Pr(Crisis|Signal)• All existing techniques such as AUROC are still
valid for examining the relative performance between the NFCI and other indicators
• The choice of threshold: based on the new utility function and the new ROC curve
![Page 14: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/14.jpg)
Not yet about forecasting
• Indicating crisis is already so difficult!• Forecasting is even more!• Alternative measure on forecasting performance
• Current: a “hit rate” measure• Alternative: Probability forecasting?
• NFCI might be linked to a good probability forecast of the distress
• Evaluating probability forecast
Always difficult to handle rare events!
![Page 15: Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De](https://reader036.vdocuments.us/reader036/viewer/2022083005/56649f2b5503460f94c4554e/html5/thumbnails/15.jpg)
Two main messages
When working on indicator/predictor of financial crisis (or any rare event)
• Should look at the right measures for evaluating the performance
• Should not be disappointed when looking at them