Beyond flat patching: wavelet tokens with cross-scale and decoupled attention for electricity load forecasting

Long-term forecasting of multivariate time series data, such as electricity demand, is crucial for managing resources, assessing risks, and monitoring environmental changes. While deep learning models have made progress in this area, they often struggle to effectively combine multi-scale temporal patterns within a single framework while also balancing the need for robust modeling and computational efficiency.

Additionally, traditional time-frequency decomposition methods often separate frequency analysis and temporal modeling, reducing the potential for unified multi-resolution learning.

To tackle these challenges, researchers have developed WCDformer (Wavelet Cross-Scale Decoupled Transformer), a new architecture that combines wavelet-based decomposition with attention mechanisms. Three main innovations underpin the WCDformer: Wavelet Multi-Resolution Tokenization uses Haar decomposition to create frequency-stratified representations; Temporal Pattern Decoupling Attention splits query formation into trend, periodic, and residual branches to process patterns more effectively; and Inter-Scale Cross-Attention enables direct information routing between different temporal scales within a single, end-to-end process.

WCDformer has been tested on five benchmark datasets, showing strong performance compared to recent advanced models while also offering a favorable balance between accuracy and computational efficiency. Stability tests on weather forecasting data indicate that some previously reported advantages may actually fall within the normal range of variation, especially in challenging situations requiring long-term predictions.

The researchers have made their approach and code publicly available, aiming to promote a more principled integration of signal processing techniques with deep learning for advanced forecasting systems.