While transformer-based foundation models excel at forecasting routine patterns, two questions remain: do they internalize semantic concepts such as market regimes, or merely fit curves? And can their internal representations be leveraged to simulate rare, high-stakes events such as market crashes? To investigate this, we introduce activation transplantation, a causal intervention that manipulates hidden states by imposing the statistical moments of one event (e.g., a historical crash) onto another (e.g., a calm period) during the forward pass. This procedure deterministically steers forecasts: injecting crash semantics induces downturn predictions, while injecting calm semantics suppresses crashes and restores stability. Beyond binary control, we find that models encode a graded notion of event severity, with the latent vector norm directly correlating with the magnitude of systemic shocks. Validated across two architecturally distinct TSFMs, Toto (decoder only) and Chronos (encoder-decoder), our results demonstrate that steerable, semantically grounded representations are a robust property of large time series transformers. Our findings provide evidence for a latent concept space that governs model predictions, shifting interpretability from post-hoc attribution to direct causal intervention, and enabling semantic what-if analysis for strategic stress-testing.
@inproceedings{sanyal2025timetime,title={time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models},author={Sanyal, Debdeep and Nagpal, Aaryan and Kumar, Dhruv and Mandal, Murari and Deshpande, Saurabh},booktitle={Recent Advances in Time Series Foundation Models Have We Reached the 'BERT Moment'?},year={2025},}
