TemporalScope Tutorial: Single-Step Target Shifting¶
Engineering Design Overview¶
The SingleStepTargetShifter class uses Narwhals for backend-agnostic target shifting in time series data, following a clear separation between validation and transformation phases:
Core Components¶
Validation Phase (fit):
- Validates TimeFrame or supported DataFrame type
- Ensures target column is set or can be inferred
- Performs input validation before any operations
Transformation Phase (transform):
- Uses Narwhals operations for backend-agnostic shifting
- Preserves TimeFrame metadata if present
- Returns consistent output across all backends
Engineering Design Assumptions¶
Single-step Mode:
- Each row represents one time step
- Target variable is shifted by specified lag
- Compatible with traditional ML frameworks
- Supports scalar target prediction tasks
Backend Agnostic:
- Uses Narwhals for all DataFrame operations
- Production environment uses pandas + narwhals
- Test environment supports multiple backends via hatch
Input Handling:
- TimeFrame: Uses existing metadata
- DataFrame: Validates in fit
- numpy array: Converts in fit
In [1]:
Copied!
import pandas as pd
import narwhals as nw
from temporalscope.core.temporal_data_loader import TimeFrame
from temporalscope.datasets.datasets import DatasetLoader
from temporalscope.target_shifters.single_step import SingleStepTargetShifter
# Load example data
loader = DatasetLoader("macrodata")
data = loader.load_data()
# Create TimeFrame
tf = TimeFrame(data, time_col="ds", target_col="realgdp")
print("Original TimeFrame:")
print(tf.df.head())
print("\n" + "-" * 80 + "\n")
# Initialize SingleStepTargetShifter (target_col inferred from TimeFrame)
shifter = SingleStepTargetShifter(n_lags=1, verbose=True)
# Transform data - target will be shifted for future prediction
transformed_tf = shifter.fit_transform(tf)
print("\nTransformed TimeFrame:")
print(transformed_tf.df.head())
print("\n" + "-" * 80 + "\n")
# Verify metadata preservation
print("TimeFrame Configuration:")
print(f"Mode: {transformed_tf.mode}")
print(f"Sort Order: {'Ascending' if transformed_tf.ascending else 'Descending'}")
import pandas as pd
import narwhals as nw
from temporalscope.core.temporal_data_loader import TimeFrame
from temporalscope.datasets.datasets import DatasetLoader
from temporalscope.target_shifters.single_step import SingleStepTargetShifter
# Load example data
loader = DatasetLoader("macrodata")
data = loader.load_data()
# Create TimeFrame
tf = TimeFrame(data, time_col="ds", target_col="realgdp")
print("Original TimeFrame:")
print(tf.df.head())
print("\n" + "-" * 80 + "\n")
# Initialize SingleStepTargetShifter (target_col inferred from TimeFrame)
shifter = SingleStepTargetShifter(n_lags=1, verbose=True)
# Transform data - target will be shifted for future prediction
transformed_tf = shifter.fit_transform(tf)
print("\nTransformed TimeFrame:")
print(transformed_tf.df.head())
print("\n" + "-" * 80 + "\n")
# Verify metadata preservation
print("TimeFrame Configuration:")
print(f"Mode: {transformed_tf.mode}")
print(f"Sort Order: {'Ascending' if transformed_tf.ascending else 'Descending'}")
======================================================================
Loading dataset: 'macrodata'
======================================================================
DataFrame shape: (203, 13)
Target column: realgdp
======================================================================
Original TimeFrame:
realgdp realcons realinv realgovt realdpi cpi m1 tbilrate \
0 2710.349 1707.4 286.898 470.045 1886.9 28.98 139.7 2.82
1 2778.801 1733.7 310.859 481.301 1919.7 29.15 141.7 3.08
2 2775.488 1751.8 289.226 491.260 1916.4 29.35 140.5 3.82
3 2785.204 1753.7 299.356 484.052 1931.3 29.37 140.0 4.33
4 2847.699 1770.5 331.722 462.199 1955.5 29.54 139.6 3.50
unemp pop infl realint ds
0 5.8 177.146 0.00 0.00 1959-01-01
1 5.1 177.830 2.34 0.74 1959-04-01
2 5.3 178.657 2.74 1.09 1959-07-01
3 5.6 179.386 0.27 4.06 1959-10-01
4 5.2 180.007 2.31 1.19 1960-01-01
--------------------------------------------------------------------------------
Initialized SingleStepTargetShifter with target_col=None, n_lags=1
Rows before: 203; Rows after: 202; Dropped: 1
Transformed TimeFrame:
realcons realinv realgovt realdpi cpi m1 tbilrate unemp \
0 1707.4 286.898 470.045 1886.9 28.98 139.7 2.82 5.8
1 1733.7 310.859 481.301 1919.7 29.15 141.7 3.08 5.1
2 1751.8 289.226 491.260 1916.4 29.35 140.5 3.82 5.3
3 1753.7 299.356 484.052 1931.3 29.37 140.0 4.33 5.6
4 1770.5 331.722 462.199 1955.5 29.54 139.6 3.50 5.2
pop infl realint ds realgdp_shift_1
0 177.146 0.00 0.00 1959-01-01 2778.801
1 177.830 2.34 0.74 1959-04-01 2775.488
2 178.657 2.74 1.09 1959-07-01 2785.204
3 179.386 0.27 4.06 1959-10-01 2847.699
4 180.007 2.31 1.19 1960-01-01 2834.390
--------------------------------------------------------------------------------
TimeFrame Configuration:
Mode: single_target
Sort Order: Ascending
Example: Using with DataFrame Directly¶
SingleStepTargetShifter also works directly with DataFrames:
In [2]:
Copied!
# Load data
data = loader.load_data()
# Initialize SingleStepTargetShifter (target_col must be specified)
shifter = SingleStepTargetShifter(target_col="realgdp", n_lags=1, verbose=True)
# Transform data
transformed = shifter.fit_transform(data)
print("Original DataFrame:")
print(data.head())
print("\n" + "-" * 80 + "\n")
print("Transformed DataFrame:")
print(transformed.head())
print("\n" + "-" * 80 + "\n")
print(f"Original shape: {data.shape}")
print(f"Transformed shape: {transformed.shape}")
print(f"Target column: {shifter.target_col}_shift_{shifter.n_lags}")
# Load data
data = loader.load_data()
# Initialize SingleStepTargetShifter (target_col must be specified)
shifter = SingleStepTargetShifter(target_col="realgdp", n_lags=1, verbose=True)
# Transform data
transformed = shifter.fit_transform(data)
print("Original DataFrame:")
print(data.head())
print("\n" + "-" * 80 + "\n")
print("Transformed DataFrame:")
print(transformed.head())
print("\n" + "-" * 80 + "\n")
print(f"Original shape: {data.shape}")
print(f"Transformed shape: {transformed.shape}")
print(f"Target column: {shifter.target_col}_shift_{shifter.n_lags}")
======================================================================
Loading dataset: 'macrodata'
======================================================================
DataFrame shape: (203, 13)
Target column: realgdp
======================================================================
Initialized SingleStepTargetShifter with target_col=realgdp, n_lags=1
Rows before: 203; Rows after: 202; Dropped: 1
Original DataFrame:
realgdp realcons realinv realgovt realdpi cpi m1 tbilrate \
0 2710.349 1707.4 286.898 470.045 1886.9 28.98 139.7 2.82
1 2778.801 1733.7 310.859 481.301 1919.7 29.15 141.7 3.08
2 2775.488 1751.8 289.226 491.260 1916.4 29.35 140.5 3.82
3 2785.204 1753.7 299.356 484.052 1931.3 29.37 140.0 4.33
4 2847.699 1770.5 331.722 462.199 1955.5 29.54 139.6 3.50
unemp pop infl realint ds
0 5.8 177.146 0.00 0.00 1959-01-01
1 5.1 177.830 2.34 0.74 1959-04-01
2 5.3 178.657 2.74 1.09 1959-07-01
3 5.6 179.386 0.27 4.06 1959-10-01
4 5.2 180.007 2.31 1.19 1960-01-01
--------------------------------------------------------------------------------
Transformed DataFrame:
realcons realinv realgovt realdpi cpi m1 tbilrate unemp \
0 1707.4 286.898 470.045 1886.9 28.98 139.7 2.82 5.8
1 1733.7 310.859 481.301 1919.7 29.15 141.7 3.08 5.1
2 1751.8 289.226 491.260 1916.4 29.35 140.5 3.82 5.3
3 1753.7 299.356 484.052 1931.3 29.37 140.0 4.33 5.6
4 1770.5 331.722 462.199 1955.5 29.54 139.6 3.50 5.2
pop infl realint ds realgdp_shift_1
0 177.146 0.00 0.00 1959-01-01 2778.801
1 177.830 2.34 0.74 1959-04-01 2775.488
2 178.657 2.74 1.09 1959-07-01 2785.204
3 179.386 0.27 4.06 1959-10-01 2847.699
4 180.007 2.31 1.19 1960-01-01 2834.390
--------------------------------------------------------------------------------
Original shape: (203, 13)
Transformed shape: (202, 13)
Target column: realgdp_shift_1
Implementation Details¶
The SingleStepTargetShifter uses Narwhals for backend-agnostic operations:
Production Environment:
- Uses pandas + narwhals for efficient operations
- Lightweight deployment with minimal dependencies
- Consistent behavior in production
Test Environment:
- Supports multiple backends via hatch
- Validates across different DataFrame implementations
- Ensures reliability across environments
Core Operations:
- Uses @nw.narwhalify for backend conversions
- Pure Narwhals operations throughout
- Consistent behavior across supported types
This design ensures reliable target shifting operations while maintaining a lightweight production footprint.
Info
This tutorial was auto-generated from the TemporalScope repository.
If you would like to suggest enhancements or report issues, please submit a Pull Request following the contribution guidelines.
Source notebook: 2_single_step_target_shifting.ipynb
Disclaimer & Copyright
THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
THIS SOFTWARE IS INTENDED FOR ACADEMIC AND INFORMATIONAL PURPOSES ONLY. IT SHOULD NOT BE USED IN PRODUCTION ENVIRONMENTS OR FOR CRITICAL DECISION-MAKING WITHOUT PROPER VALIDATION. ANY USE OF THIS SOFTWARE IS AT THE USER'S OWN RISK.
© 2024 Philip Ndikum