I've now written about two approaches to detecting regime changes in financial time series: MMD, which makes no assumptions about the data but looks backward over windows, and BOCPD, which assumes a distributional model but detects changes in real time. Each post explored one method in isolation.
The obvious next question: how do they actually compare on the same data?
The Experiment
To separate two distinct questions—does the method matter? and do the features matter?—I ran both methods in a 2x2 design:
|
Univariate (log returns) |
Multivariate (log OHLCV, 5 features) |
| BOCPD |
Gaussian model on returns |
Gaussian model on 5 features |
| MMD |
RBF kernel on returns |
RBF kernel on 5 features |
Same data (SPY daily prices, 2018–2024), same feature engineering, same known market events used for validation. The only things that change are the detection method and the number of features. Both methods share a common data pipeline built on finfeatures, a package I built for fetching and transforming OHLCV data.
Side by Side on Real Data
Here's what both methods find when applied to the same SPY data:
The top panel tells the story at a glance. BOCPD multivariate (solid orange) detects 4 boundaries. MMD multivariate (solid purple) detects 43. That's not a typo—same data, same features, an order-of-magnitude difference in detections.
The middle panels show why. MMD's z-score (second panel) spikes frequently above the detection threshold, particularly with multivariate features—nearly every window comparison finds a statistically significant distributional difference. BOCPD's expected run length (third panel) drops only at the sharpest transitions, remaining stable through periods where MMD sees change everywhere.
The bottom panel shows the known market events we're validating against: the COVID crash and recovery (2020), the 2022 drawdown and bottom, and the 2023 acceleration.
The Feature Effect: Same Data, Opposite Results
This is the most striking finding. Adding multivariate features has opposite effects on the two methods:
- BOCPD goes from 9 boundaries to 4. The multivariate Gaussian model has to explain covariance across 5 dimensions. Only the clearest distributional shifts survive that higher bar—the model becomes more conservative.
- MMD goes from 1 boundary to 43. The RBF kernel naturally captures joint distributional differences across all features. More features means more surface area for the test to find significant differences—the method becomes more sensitive.
The same features make a parametric model more conservative and a nonparametric test more liberal. This asymmetry wasn't something I expected going in, and it has real practical implications: your choice of features doesn't just affect what you detect—it changes the fundamental character of each method's output.
Which Events Do They Catch?
Against the five known market events, MMD multivariate is the only configuration that detects all five, with offsets generally under 20 trading days. BOCPD multivariate catches 4 of 5 (missing the 2022 bottom). BOCPD univariate catches only 2 of 5, and MMD univariate catches just 1.
The lesson: multivariate features help both methods in absolute terms, but MMD benefits dramatically more. If your goal is to catch every event and you'll filter false positives later, MMD multivariate is the strongest choice. If you want fewer, higher-confidence detections, BOCPD multivariate is more selective.
Two Signals, Not One
Rather than choosing a winner, it's worth looking at both signals together:
Both signals peak during the same major events—the COVID crash, the 2022 drawdown—but they get there differently. BOCPD reacts sharply to individual observations, spiking the moment something unusual arrives. MMD builds gradually across a window, producing a smoother signal that crests as the full window fills with post-change data.
They're complementary rather than redundant. A spike in BOCPD without a corresponding MMD rise might be a single outlier. A sustained MMD elevation without a BOCPD reaction might be a subtle drift that doesn't violate the Gaussian model. Agreement between both is the strongest signal.
Testing on Controlled Data
Real markets don't come with ground truth, so I generated synthetic data with planted change points to measure precision and recall directly.
When the Gaussian assumption holds, BOCPD has higher recall—it catches more of the planted shifts, particularly variance changes where MMD struggles. But it comes with more false positives.
BOCPD achieves 0.93 recall on mean shifts and 0.97 on variance shifts, versus MMD's 0.78 and 0.38 respectively. MMD nearly never fires a false alarm (precision ~1.0), while BOCPD averages 1–2 false positives per run. The variance shift result is particularly notable—MMD's kernel bandwidth, tuned to the overall data scale, is much less sensitive to scale-only changes than BOCPD's explicit variance tracking.
When the data has fat tails, the picture inverts:
As tails get heavier (moving from df=30 toward df=3), BOCPD's precision drops from 0.73 to 0.18 and recall from 0.88 to 0.25. The Gaussian model interprets heavy-tailed outliers as evidence of regime changes, generating ~5 false positives per run. MMD stays rock-solid throughout—precision stays at 1.0 and recall holds at 0.70–0.75 regardless of tail behavior.
This also shows up in BOCPD's credible intervals:
Under correct specification, the 90% credible intervals actually contain the true change point about 90% of the time. Under heavy tails (df=3), coverage drops to 62%. The uncertainty estimates that are one of BOCPD's key selling points become unreliable exactly when you need them most.
Tuning Sensitivity
Both methods have knobs, and they respond differently to tuning:
MMD's boundary count scales roughly linearly with window size—from 25 detections at window=20 to 74 at window=60. This means the choice of window size substantially controls how many boundaries you get. BOCPD's hazard rate sweep tells the opposite story: 4–5 boundaries across the full range tested. The posterior updates dominate the prior, making BOCPD's output remarkably stable to this particular tuning choice.
When to Use Which
Use BOCPD for real-time monitoring. It processes one observation at a time, runs fast (under 1 second for 1500 observations with multivariate features), and gives you calibrated uncertainty when its assumptions hold. Best for streaming data where you need to act on detections as they arrive.
Use MMD for offline review. It's robust to non-Gaussian data, catches subtle multivariate shifts that BOCPD misses, and never generates false positives in controlled tests. The computational cost is higher (permutation testing), but that matters less when you're analyzing historical data.
The best strategy may be to use both. BOCPD for real-time alerting with high specificity, MMD for periodic offline review with high sensitivity. When both methods agree, you can be most confident a genuine regime change occurred.
Future Work
There are several directions worth exploring from here. Other regime detection approaches—Hidden Markov Models, PELT, Bayesian nonparametric methods—bring their own tradeoffs and could extend this comparison. Combining the two signals into a single detection framework, rather than running them independently, could capture the best of both. And as noted in the BOCPD post, the package now includes Student-t observation models that directly address the heavy-tail limitation—rerunning these comparisons with those models would test whether BOCPD can match MMD's robustness while keeping its online advantages.
Code and Resources
The full comparison is available on GitHub: regime-detection-comparison
The repository includes rendered notebooks with full-resolution figures and complete code for both the SPY comparison and the synthetic experiments.
The two methods compared here are implemented as separate packages:
References
- Adams, R. P., & MacKay, D. J. C. (2007). Bayesian Online Changepoint Detection. arXiv:0710.3742.
- Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., & Smola, A. (2012). A Kernel Two-Sample Test. JMLR, 13:723–773.
- Whitham-Powell, E. (2025). Detecting Market Regime Changes with Maximum Mean Discrepancy. whitham-powell.com.
- Whitham-Powell, E. (2026). Detecting Market Regime Changes with Bayesian Online Change Point Detection. whitham-powell.com.