@rodionov2004: A Sequential Algorithm for Testing Climate Regime Shifts

An algorithm to detect climate regime shifts. Classic work, widely cited.

paper scope/work/research/climateRegimeShift scope/work/research/algorithm

citation_key: rodionov2004
title: A Sequential Algorithm for Testing Climate Regime Shifts
author: Sergei N. Rodionov
journal: Geophysical Research Letters
year: 2004
doi: 10.1029/2004GL019448

Overview

No gonna describe it in great detail.

First, it's sequential, meaning it processes the time series in order, and different start points can lead to different results.
Second, it processes new data points as they come in. Every new data is checked immediately. so it's kinda near-real-time.
Third, since it deals the data points in a specific window, abnormal values won't have a decisive influence on the results.

Algo accepts two parameters: cut-off length $l$ and significance level $p$ .

Breif Intro

Calculate $d i f f$ according to the series and $l$ : $\begin{matrix} (1) & d i f f = t \sqrt{2 σ_{l}^{2} / l} \end{matrix}$ where $t$ is the critical value of the Student's t distribution with $2 l - 2$ degrees of freedom at the significance level $p$ (two-tailed), and $σ_{l}^{2}$ is the average variance of the series in running windows of length $l$ .
Calculate the ${\overset{―}{x}}_{R k}$ where $R i$ is the $k$ -th regime. Initially, $R 1$ is the first $l$ data points.
For each new data point $x_{i}$ where $i$ starts from $l + 1$ :
1. If $| {\overset{―}{x}}_{R k} - x_{i} | > d i f f$ , then a new possible regime is detected. The possible new regime $R_{k + 1}$ starts from $x_{i}$ . Move to 5.
2. Otherwise, $x_{i}$ is not significantly different from the current regime, meaning the current regime continues, and we should update ${\overset{―}{x}}_{R k}$ by including $x_{i}$ . Move back to 3 to check the next data point.
For the new possible regime $R_{k + 1}$ which starts from $i$ , calculate the $R S I_{i, i}$ to $R S I_{i + l - 1, i}$ . $R S I$ is calculated as: $\begin{matrix} (2) & R S I_{i, j} = \sum_{i = j}^{j + m} \frac{x_{i}^{*}}{l σ_{l}}, m = 0, 1, \dots, l - 1 \end{matrix}$ where $x_{i}^{*} = x_{i} - {\overset{―}{x}}_{R k} - d i f f$ if the shift is up, $x_{i}^{*} = {\overset{―}{x}}_{R k} - x_{i} - d i f f$ if the shift is down.
1. If any of them is negetive, then the possible new regime is rejected. Update the current regime $R_{k}$ by including $x_{i}$ and move back to 3 to check the next data point.
2. Otherwise, all the $R S I$ s are positive, then the possible new regime is accepted. New regime $R_{k + 1}$ starts from $x_{i}$ and move back to 3 to check the next data point.

Note that once a new regime is detected, the next to-be-checked data point is the next point of the regime shift point, no plus $l$ . For instance, if the 1950 is detected as a regine shift, meaning $R S I_{1950, 1950}$ to $R S I_{1959, 1950}$ are all positive, then the next to-be-checked data point is still 1951, not 1960. This simply means that a regime is not necessarily $l$ long, it can be shorter, and the next regime can start at any point after the shift point.

However, the shift point is usually unlikely to exist in the first $l$ data points after previous shift point, because points in $l$ length window are not likely to be significantly different from the current regime.