The credit gap still warns of banking crises. The low-noise signal does not reproduce.

What was run

The indicator is the BIS’s own published credit-to-GDP gap: the ratio of credit to the private non-financial sector to GDP, minus its long-run trend from a one-sided (backward-looking) Hodrick–Prescott filter, in percentage points of the ratio. That construction is the BIS’s stated one for the published series, and the Basel III buffer guide specifies lambda 400,000 for this gap (BCBS, December 2010). It is not a growth rate; the same file carries the ratio itself under a different dtype, which is not used here. The quarterly series is reduced to one year-end value per country-year, matching the paper’s annual frequency: 1,897 observations for 43 economies, 1957–2025. The file’s 1 aggregate without a country code (Euro area) is excluded, because the crisis chronology dates crises for countries.

The outcome is a banking-crisis onset in the Laeven–Valencia 2026 chronology: 164 crises, first onset 1976, last 2023. A country-year is a positive if an onset falls one to three years ahead, the paper’s “years ahead” framing. Country-years inside an ongoing crisis (onset year through the chronology’s end year; the onset year alone where no end year is recorded) are dropped, so post-crisis gap collapses do not count as false alarms. Country-years after 2022 are dropped because their three-year window extends past the data. That leaves 1,650 country-years: 103 positives and 1,547 negatives, a base rate of 6.2%.

The direction reproduces

Ranking all 1,650 country-years by the gap gives an AUC of 0.661: the probability that a randomly drawn pre-crisis year carries a higher gap than a randomly drawn quiet year. The Hanley–McNeil standard error is 0.030, putting the estimate 5.4 standard errors above the 0.5 of an uninformative indicator. That standard error assumes independent observations, which these are not: the 103 positives come from 35 crises, each contributing up to three pre-crisis years. Read 0.030 as a lower bound on the true uncertainty; no clustered standard error is computed here.

The noise does not

The paper’s case was not the direction; it was that the signal is clean enough to act on. In the table, TPR is the share of the 103 pre-crisis country-years with a gap above the threshold, FPR the share of the 1,547 quiet ones, and noise-to-signal is FPR/TPR, the paper’s own measure. None of the four thresholds is ours: 2 and 10 are the L and H anchors of the Basel III countercyclical buffer guide (BCBS, December 2010), which was built on this indicator; 4 is the value the paper judged best; 6 sits in the paper’s published table range.

Source: BIS credit-to-GDP gap series (bis_credit_gap.parquet), gap dtype: actual ratio minus one-sided HP trend, year-end value per country-year. Laeven and Valencia (2026), Systemic Banking Crises Database 1970-2025, IMF Working Paper WP/26/94 (laeven_valencia_2025.parquet). Borio and Lowe (2002), BIS Working Paper 114, Table 1, panel C (horizon = 3 years), credit-gap columns; the paper's own published figures. The paper's table starts at a 3-point threshold, so it has no entry at 2. Its convention counts a signal first issued in the crisis year itself as a success; ours requires the signal one to three years before onset.

The best value in our column, 0.431 at a 10-point gap, is worse than the worst credit-gap value anywhere in the paper’s three-year table, 0.25 at a 3-point gap. At the paper’s one-year horizon and 4-point threshold, the paper reported 0.24 with 79 percent of crises caught (its own figures). Raising the threshold to the Basel cap of 10 buys precision at the price the paper’s own table shows: our TPR falls from 0.583 to 0.369.

Crisis by crisis: who the gap saw coming

35 of the 164 Laeven–Valencia crises have at least one gap observation in the three years before onset. Of those 35, 27 carried a gap above 2 in at least one pre-crisis year and 17 carried a gap above 10. The remaining 8 began with every observed pre-crisis gap negative: Netherlands 2008 (t-1 gap -13.6), Japan 1997 (t-1 gap -13.0), Hungary 1991 (t-1 gap -10.0), Germany 2008 (t-1 gap -9.7), India 1993 (t-1 gap -4.6), Austria 2008 (t-1 gap -3.0), Thailand 1983 (t-1 gap -3.0), Switzerland 2008 (t-1 gap -1.1). 4 of the 8 are 2008 onsets: Netherlands, Switzerland, Austria, Germany, crises the chronology dates at the height of the global wave while each country’s own gap was negative. A domestic credit gap cannot flag a crisis that arrives without a domestic credit boom, and with every observed pre-crisis gap negative, each of these is a guaranteed miss at every positive threshold.

The top of the chart needs a flag: Luxembourg’s 90.0 gap in 2007 sits on a published series that begins in 2005-Q1. The BIS publishes the value as a normal observation (obs_status A) and it is kept here, not trimmed.

One pre-declared robustness cut

One variant was declared before running: restrict the panel to the chronology window 1970–2020, where the upper bound is the last recorded onset year minus the three-year horizon, so that every negative is verifiable against the chronology. On those 1,537 country-years (103 positives, base rate 6.7%), the AUC is 0.651 (SE 0.030), and the threshold column above moves the way the table shows. The main result is not an artifact of the pre-1970 observations or of the censored tail.

Why this diverges from the paper

The shortfall is not only against the 2002 paper. Drehmann and Juselius (2014), evaluating the same indicator on 26 economies with quarterly data and a different crisis list, report credit-gap AUCs between 0.83 and 0.85 over the first four years of their forecast horizons (their figures). Ours is 0.661. Four measurable differences separate this run from those:

1. The crisis chronology. The paper dated its episodes with Bordo et al (2001); this note uses Laeven–Valencia 2026, which scores Netherlands, Switzerland, Austria, Germany in 2008 as systemic banking crises. Each entered with every observed pre-crisis gap negative, so each is an automatic miss that a chronology classifying them differently would not charge to the indicator.
2. Sample composition. 18 of the 35 in-coverage crises are 2007–08 onsets, effectively one global event. The paper’s episodes all fall in 1970–1999 (its own window), and 102 of the 129 crises this panel cannot see began before 2000.
3. Vintage and filter. The paper computed its own gaps from annual 1960–1999 data with an ex ante rolling HP filter, on credit series as published in 2002. Here the input is the BIS’s currently published quarterly gap: one-sided HP, lambda 400,000, credit and GDP as revised today.
4. The post-2010 panel. The paper’s data end in 1999; this panel runs to 2022. At the 10-point threshold, 88 of the 246 false-positive country-years fall in 2010 or later (6 of them China): gaps above 10 with no onset within three years in the chronology, each of which also counts against every lower threshold.

What this cannot tell you

• The country-year n overstates the evidence. 103 positive country-years come from 35 crises, and 18 of those crises are 2007–08 onsets. The crisis-level counts (27 of 35 above 2, 17 of 35 above 10) are the honest unit; the AUC’s standard error of 0.030 is a lower bound because the observations it assumes independent are not.
• Coverage is selected. 129 of the 164 Laeven–Valencia banking crises have no pre-crisis gap observation at all, because the country is not in the BIS’s 43-economy set or its series starts too late; 102 of the 129 began before 2000. They are absent from the panel, not scored as anything, and this page can say nothing about how the gap would have performed on them.
• This is not the authors’ dataset. What is tested here is the published-indicator version of the claim: the series the BIS computes today, against a 2026 crisis chronology. The paper’s own 34-country annual dataset, its Bordo et al crisis dates, and its self-computed gaps are not in this estate.
• The annualisation is a choice. One year-end value per country-year matches the paper’s annual frequency. A quarterly panel would multiply the observation count roughly fourfold with near-duplicate information and make the independence problem worse, not better.
• The censored tail is only partly handled. The main panel drops country-years after 2022, but its negatives in the final years are still judged against a chronology whose last recorded onset is 2023; a crisis dated after that would flip some of them. The robustness bound of 2020 removes the issue, and the AUC moves from 0.661 to 0.651.