PatchTSMixer in HuggingFace

Back to Articles PatchTSMixer in HuggingFace - Getting Started Published January 19, 2024 Update on GitHub Upvote 10 +4 Arindam Jati ajati Follow guest Vijay Ekambaram vijaye12 Follow guest Nam Nguyen namctin Follow guest Wesley M. Gifford wmgifford Follow guest Kashif Rasul kashif Follow Niels Rogge nielsr Follow PatchTSMixer is a lightweight time-series modeling approach based on the MLP-Mixer architecture. It is proposed in TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting by IBM Research authors Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong and Jayant Kalagnanam. For effective mindshare and to promote open-sourcing - IBM Research joins hands with the HuggingFace team to release this model in the Transformers library. In the Hugging Face implementation, we provide PatchTSMixer’s capabilities to effortlessly facilitate lightweight mixing across patches, channels, and hidden features for effective multivariate time-series modeling. It also supports various attention mechanisms starting from simple gated attention to more complex self-attention blocks that can be customized accordingly. The model can be pretrained and subsequently used for various downstream tasks such as forecasting, classification, and regression. PatchTSMixer outperforms state-of-the-art MLP and Transformer models in forecasting by a considerable margin of 8-60%. It also outperforms the latest strong benchmarks of Patch-Transformer models (by 1-2%) with a sign...