Out-of-Distribution Generalization in Time Series

Built and currently maintained by Xin Wu.

Overview

A list of research papers on out-of-distribution (OOD) generalization in time series. Existing studies categorize the problem from three key perspectives: data distribution, representation learning, and OOD evaluation. For more details, please refer to our survey paper, “Out-of-Distribution Generalization in Time Series: A Survey.”

Data Distribution

Real-world data distributions are often dynamic rather than static and frequently subject to various distribution shifts that challenge the assumptions made during training. Two common distribution shifts are covariate and concept shifts.

Covariate Shift

  • [CVPR 2021] Out-of-distribution Detection and Generation using Soft Brownian Offset Sampling and Autoencoders [Paper] [Code]
  • [ICLR 2022] Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift [Paper] [Code]
  • [SMC 2022] Feature Importance Identification for Time Series Classifiers [Paper]
  • [FUZZ 2023] An Initial Step Towards Stable Explanations for Multivariate Time Series Classifiers with LIME [Paper]
  • [Inf. Sci. 2023] Explaining Time Series Classifiers through Meaningful Perturbation and Optimisation [Paper] [Code]
  • [J.Neunet 2024] SEGAL Time Series Classification - Stable Explanations Using A Generative Model and An Adaptive Weighting Method for LIME [Paper]
  • [VR 2024] Generating Virtual Reality Stroke Gesture Data from Out-of-Distribution Desktop Stroke Gesture Data [Paper] [Code]
  • [AAAI 2024] Generalizing across Temporal Domains with Koopman Operators [Paper] [Code]
  • [IJCAI 2024] Temporal Domain Generalization via Learning Instance-level Evolving Patterns [Paper] [Code]
  • [ICML 2024] Connect later: Improving fine-tuning for robustness with targeted augmentations [Paper]
  • [ICML 2024] TimeX++: Learning Time-Series Explanations with Information Bottleneck [Paper] [Code]

Concept Shift

  • [AAMAS 2024] Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection [Paper] [Code]
  • [ICLR 2024] Disentangling Time Series Representations via Contrastive Independence-of-Support on l-Variational Inference [Paper] [Code]

Representation learning

Representation learning enhances generalization by extracting robust and generalizable features. These methods can be categorized into four groups: decoupling-based models, invariance-based models, ensemble-based learning, and large time-series models.

Decoupling-based Methods

Multi-Structured Analysis:

  • [NeurIPS 2020] Feature Shift Detection: Localizing Which Features Have Shifted via Conditional Distribution Tests [Paper] [Code]
  • [IJCNN 2021] Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine [Paper] [Code]
  • [ACM TCPS 2022] Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systems [Paper] [Code]
  • [ACM TIST 2023] Out-of-distribution Detection in Time-series Domain: A Novel Seasonal Ratio Scoring Approach [Paper] [Code]
  • [KDD 2024] Orthogonality Matters: Invariant Time Series Representation for Out-of-distribution Classification [Paper] [Code]
  • [TKDE 2024] Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting [Paper] [Code]
  • [AAAI 2024] MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecasting [Paper] [Code]

Causality-Inspired:

  • [CVPR 2021] Causal Hidden Markov Model for Time Series Disease Forecasting [Paper] [Code]
  • [ICRA 2022] Causal-based Time Series Domain Generalization for Vehicle Intention Prediction [Paper]
  • [ICML 2023] Neural Stochastic Differential Games for Time-series Analysis [Paper] [Code]
  • [Sci. Robot. 2023] Robust Flight Navigation Out of Distribution with Liquid Neural Networks [Paper] [Code]
  • [Inf. Sci. 2024] A Causal Representation Learning Based Model for Time Series Prediction under External Interference [Paper] [Code]

Invariant-based Methods

Invariant Risk Minimization Models:

  • [AAAI 2021] Meta-learning Framework with Applications to Zero-shot Time-series Forecasting [Paper]
  • [ICRA 2023] Robust Forecasting for Robotic Control: A Game-Theoretic Approach [Paper]
  • [KDD 2023] DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecasting [Paper] [Code]
  • [KDD 2023] TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting [Paper]
  • [WWW 2024] Towards Invariant Time Series Forecasting in Smart Cities [Paper] [Video]
  • [ICML 2024] Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning [Paper] [Code]
  • [ICLR 2025] Out-of-distribution Generalization for Total Variation based Invariant Risk Minimization [Paper] [Code]

Domain-Invariance Methods:

  • [ICLR 2021] In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness [Paper] [Code]
  • [AAAI 2021] Latent independent excitation for generalizable sensor-based cross-person activity recognition [Paper] [Code]
  • [NeurIPS 2023] Evolving Standardization for Continual Domain Generalization over Temporal Drift [Paper] [Code]
  • [ICLR 2023] Out-of-distribution Representation Learning for Time Series Classification [Paper] [Code]
  • [VR 2024] Generating Virtual Reality Stroke Gesture Data from Out-of-Distribution Desktop Stroke Gesture Data [Paper] [Code]
  • [AAAI 2024] Generalizing across Temporal Domains with Koopman Operators [Paper] [Code]
  • [ICML 2024] Connect later: Improving fine-tuning for robustness with targeted augmentations [Paper]
  • [NeurIPS 2024] Continuous Temporal Domain Generalization [Paper] [Code]
  • [TPAMI 2024] Diversify: A General Framework for Time Series Out-of-Distribution Detection and Generalization [Paper] [Code]
  • [Struct. 2024] Enhancing Time Series Data Classification for Structural Damage Detection through Out-of-Distribution Representation Learning [Paper]
  • [DAC 2024] SMORE: Similarity-based Hyperdimensional Domain Adaptation for Multi-Sensor Time Series Classification [Paper] [PPT]

Ensemble-based Learning

  • [SAIS 2022] Out-of-distribution in Human Activity Recognition [Paper]
  • [RESS 2022] Out-of-Distribution Detection-Assisted Trustworthy Machinery Fault Diagnosis Approach with Uncertainty-Aware Deep Ensembles [Paper]
  • [KDD 2023] Maintaining the Status Quo: Capturing Invariant Relations for OOD Spatiotemporal Learning [Paper] [Code]
  • [ICC 2023] Out-of-distribution Internet Traffic Prediction Generalization Using Deep Sequence Model [Paper]
  • [AIC 2023] Classifying Falls Using Out-of-Distribution Detection in Human Activity Recognition [Paper]
  • [AAMAS 2024] Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection [Paper] [Code]

Large Time-Series Models

Tuning-based Methods:

  • [NeurIPS 2023] ForecastPFN: Synthetically-Trained Zero-Shot Forecasting [Paper] [Code]
  • [EACL 2023] Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? [Paper] [Code]
  • [NeurIPS 2024] Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-shot Forecasting of Multivariate Time Series [Paper] [Code]
  • [NeurIPS 2023] JoLT: Jointly Learned Representations of Language and Time-Series [Paper]
  • [ICASSP 2024] ETP: Learning Transferable ECG Representations via ECG-Text Pre-Training [Paper]
  • [AAAI 2024] JoLT: Jointly Learned Representations of Language and Time-Series for Clinical Time-Series Interpretation (Student Abstract) [Paper]
  • [ICLR 2024] Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [Paper] [Code]
  • [ICML 2024] Unified Rraining of Universal Time Series Forecasting Transformers [Paper] [Code]
  • [CIKM 2024] General Time Transformer: an Encoder-only Foundation Model for Zero-Shot Multivariate Time Series Forecasting [Paper] [Code]
  • [NeurIPS 2024] Align and Fine-Tune: Enhancing LLMs for Time-Series Forecasting [Paper]
  • [AAAI 2025] ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data [Paper] [Code]
  • [AAAI 2025] CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning [Paper] [Code]

Non-tuning-based Methods:

  • [NeurIPS 2023] Large Language Models Are Zero-Shot Time Series Forecasters [Paper] [Code]
  • [ArXiv 2023] Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain [Paper] [Code]
  • [ArXiv 2023] TimeGPT-1 [Paper] [Code]
  • [ArXiv 2024] TableTime: Reformulating Time Series Classification as Training-Free Table Understanding with Large Language Models [Paper] [Code]

Full-training-based (training from scratch):

  • [NeurIPS 2023] One Fits All: Power General Time Series Analysis by Pretrained LM [Paper] [Code]
  • [R0-FoMo 2023] Lag-Llama: Towards Foundation Models for Time Series Forecasting [Paper] [Code]
  • [ICML 2024] MOMENT: A Family of Open Time-series Foundation Models [Paper] [Code]
  • [ICML 2024] Timer: Generative Pre-trained Transformers Are Large Time Series Models [Paper] [Code]
  • [ICML 2024] A Decoder-only Foundation Model for Time-series Forecasting [Paper] [Code]
  • [TMLR 2024] Chronos: Learning the Language of Time Series [Paper] [Code]
  • [ICLR 2025] Towards Neural Scaling Laws for Time Series Foundation Models [Paper] [Code]

Time Series Datasets 📊

📌 General Time Series

  • 💾 UEA Classification – A rich set of time series datasets for classification tasks.
  • 📊 UCR Archive – Benchmark datasets for time series research.

💰 Economy & Finance

  • 📈 PPA Dataset – Causal inference data for Power Purchase Agreements.
  • 💹 Exchange Rate – Historical exchange rates for financial forecasting.

🚗 Transportation

  • 🛣 SIP Traffic – Traffic data from Suzhou Industrial Park.
  • 🚌 METR-LA – Los Angeles traffic sensor data.
  • ✈️ Flight Data – Aircraft trajectory & air traffic. data.
  • 🚦 Traffic Flow – Multi-city traffic congestion datasets.

⚡ Energy & Environment

🌫 Air Quality

  • 🌍 KnowAir PM2.5 – Air pollution dataset focusing on PM2.5 levels.

🏥 Health & Medicine

  • 🦾 EMG - Electromyography dataset records muscle activity.

⌚️ Human Activity Recognition

  • 🏃🏻‍♀️ UCIHAR - Smartphone-based human activity recognition dataset.
  • 🩼 UniMiB SHAR - Smartphone-based human activity recognition and fall detection dataset.
  • 📱 Opportunity - Multiple wearable and environmental sensors for human activity recognition dataset.

🛠 More Datasets

  • [ArXiv 2021] Towards Out-of-Distribution Generalization: A Survey [Paper]
  • [ArXiv 2022] Out-of-Distribution Generalization on Graphs: A Survey [Paper]
  • [EMNLP 2023] Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future [Paper]
  • [TPAMI 2024] Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects [Paper]
  • [TPAMI 2024] A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection [Paper]
  • [TKDE 2024] A Survey on Time-Series Pre-Trained models [Paper]
  • [TMLR 2024] WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series [Paper]

Acknowledgement

Last updated on March 7, 2025. (For problems, contact xinwu5386@gmail.com. To add papers, please pull request at our repo)