Evaluate a forecast with time-ordered splits

A forecast evaluation should train on information that existed before its test period. Randomly mixing a dated series can let later observations influence a model tested on earlier dates. Inspect the split boundaries before comparing scores, and fit preprocessing inside each training window.

Check chronological folds

This synthetic example uses twelve equally spaced observations. It checks the ordering of three expanding training windows and leaves one sample between training and testing.

from sklearn.model_selection import TimeSeriesSplit

observations = list(range(12))
splitter = TimeSeriesSplit(n_splits=3, test_size=2, gap=1)
folds = list(splitter.split(observations))
assert len(folds) == 3
for train, test in folds:
    assert train[-1] < test[0]
    assert test[0] - train[-1] == 2
    assert len(test) == 2

The assertions check index boundaries, not forecast quality. A real dataset needs correctly ordered timestamps and a policy for missing or duplicate observations. Scikit-learn notes that equal spacing is needed for comparable time durations across these folds.

Training ends before the excluded gap and later test window; preprocessing must respect the same boundary.

Match the gap to the prediction problem

Suppose each label summarizes activity over the following week. Adjacent rows can then share future information even though their timestamps differ. A gap measured in samples should reflect that label overlap and the dataset's sampling interval. Setting an arbitrary gap of one does not resolve every leakage problem.

Feature generation can leak information too. A centered rolling average uses future observations, and scaling on the entire series exposes test-period statistics. Build each fold's transformations from its training data, then apply the fitted transformation to the later test data. Save that sequence with the experiment so that a score can be reproduced.

Compare a simple baseline

Evaluate a deliberately simple predictor, such as the last observed value, on the same folds and targets. Report the error definition, horizon and number of matched observations with each result. Aggregate scores can hide a poor later period, so inspect fold results separately before combining them.

This educational fixture uses synthetic integers and makes no claim about a portfolio project's forecast accuracy or financial performance. It also does not decide whether a model should be deployed. Reference: TimeSeriesSplit documentation. The JSLL project overview provides related engineering context, with its own evidence boundaries.