sliding_window
TemporalScope/src/temporalscope/partition/single_target/sliding_window.py
This module defines the SlidingWindowPartitioner class, a specific implementation of the TemporalPartitionerProtocol for creating contiguous, non-overlapping partitions using a sliding window mechanism.
The SlidingWindowPartitioner divides a dataset into non-overlapping partitions using a fixed window size and optional stride. The stride determines how far to move between the starting points of consecutive partitions, which can introduce gaps between them. Each partition can be further split into train, test, and validation sets.
This class utilizes the generator pattern for memory efficiency, yielding
partition indices and data slices one at a time. The SlidingWindowPartitioner
is intended for universal models, which assume flat partitioning across all
entities. Users are responsible for preprocessing steps such as deduplication or
transforming time_col to numerical features.
Engineering Design:
| Aspect | Description |
|---|---|
| Partial Temporal Ordering | Follows a universal model design; allows overlapping labels within partitions while leaving strict temporal ordering to the user. |
| Narwhals API | Leverages Narwhals backend for efficient operations. Users can switch between supported backends (e.g., Pandas, Polars, Modin) using core_utils. |
| Dataset Accessibility | Inspired by Dask and TensorFlow design. Provides hierarchical access via partitions[index]["train"], test, validation, or full labels. |
| Lazy/Eager Execution | Narwhals backend supports lazy/eager evaluation; generator pattern ensures memory-efficient fit and transform operations. |
| Human-Centric Design | Combines human-readable labels (train, test) with indexing for scalable workflows, reducing cognitive overhead when handling large numbers of partitions. |
| Padding Control | Leaves padding decisions (e.g., zero-padding) to users while allowing configurable truncation. |
Visualization:
The table below illustrates how the sliding window mechanism works with overlapping partitions. Each "X" represents a row included in the respective partition, based on the configured window size and stride.
| Time | Partition 1 | Partition 2 | Partition 3 | Partition 4 | Partition 5 |
|---|---|---|---|---|---|
| 2021-01-01 | X | ||||
| 2021-01-02 | X | X | |||
| 2021-01-03 | X | X | |||
| 2021-01-04 | X | X | |||
| 2021-01-05 | X | X | |||
| 2021-01-06 | X |
See Also
- Gu et al., 2021. The sliding window and SHAP theory applied to long short-term memory networks for state of charge prediction.
- Pham et al., 2023. Speech emotion recognition using overlapping sliding window and explainable neural networks.
- Van Zyl et al., 2024. Explainable AI for feature selection in time series energy forecasting with Grad-CAM and SHAP.
- Bi et al., 2020. Prediction model for identifying methylation sites with XGBoost and SHAP explainability.
- Zimmermann et al., 2022. Improving drift detection by monitoring SHAP loss values in pattern recognition workflows.
- Li et al., 2022. Visualizing distributional shifts using SHAP in machine learning models.
- Seiffer et al., 2021. Concept drift detection in manufacturing data with SHAP for error prediction improvement.
- Haug et al., 2022. Change detection for local explainability in evolving data streams.
- Zhao et al., 2020. Feature drift detection in evolving data streams with database applications.