@satish

Noticed a lot of things broke after the recent foundation change from 3 State HMM to 4 State.

Started by profiling the research pipeline with py-spy. Rewrote the feature engineering loop using vectorized NumPy operations and numba.jit for the rolling entropy calculations. Cut preprocessing time from 22min to 6min using better caching routes. Also fixed a silent type coercion bug that was casting ints to floats incorrectly.

Fixed several issues that surfaced after the 4-state HMM migration. The dashboard is now stable across all 8 assets. Fixed SPY regime forecasts showing negative probabilities by adding eigen-decomposition stability checks before forecast generation. Resolved Chrome/Recharts memory leak on the comparison tab by switching to lazy loading for asset comparison charts, reducing memory usage by ~40%. Fixed crypto state collapse on volatile days by increasing minimum state occupancy to 5% and re-running with 60 random seeds. Resolved 404 errors on the SHAP endpoint for IWM by fixing feature extraction in the KernelExplainer. Extended Yahoo Finance cache TTL to 7200s with fallback to previous day’s data when rate-limited. Fixed regime confidence display on the live tab by correcting posterior probability parsing. All 8 assets now load correctly across all 3 time periods. Next up: multi-asset portfolio execution endpoint and backtest optimization for 1-year periods.

Migrated Regim’s core regime detection from 3-state
to 4-state HMM.

This required substantial refactoring of the backend: adding a 4th state (distinguishing “Crash” from “Bearish”), updating all 11 API endpoints to handle the new state space, re-validating
statistical tests, and fixing cascading issues with backtest accuracy, regime labeling, and volatility forecasting.