Level 3 AI in trading · Part 5/8

Overlapping Examples and Sample Weights

Almost every standard machine learning method assumes training examples are drawn independently. On financial data that assumption is nowhere near satisfied – and the violation is not small but dominant.

Where the dependence comes from

If a label spans the next ten periods, two neighbouring examples share nine of those ten. Their outcomes are determined by largely the same price moves.

To a model that looks like confirmation: the same relationship, again and again. In reality it is a single observation entered ten times. The effective sample size is therefore drastically smaller than the row count – and confidence statements based on the row count are correspondingly too tight.

Uniqueness

The remedy starts with a measurement. For each point in time, count how many labels cover it simultaneously. An example whose outcome window overlaps many others is barely unique; one standing alone is fully unique.

Average uniqueness across its window gives every example a weight. During training, heavily overlapping examples then count for less. The effect is unspectacular and important: the model stops mistaking repetition for evidence.

Weighting by outcome magnitude

A second, independent weighting: examples whose associated move was large carry more information than those that barely cleared the noise. A common choice weights by absolute return over the outcome window, divided by the number of concurrently running labels.

Together – uniqueness and magnitude – this brings training closer to what matters economically.

Time decay

Markets change. A relationship from 2011 is not wrong, but probably less relevant than one from last year. A linearly or exponentially decaying time weight accounts for that.

The setting is a trade-off, not an optimisation problem: strong decay makes the model adaptive and unstable, weak decay stable and sluggish. What matters is only that the decision is made deliberately and not tuned on the test data – otherwise it becomes the next way to burn through it.

Sequential bootstrap

Bagging methods such as random forests draw training examples at random with replacement. With overlapping data, nearly identical examples land in the same draw, and the trees of a forest become more similar than intended – the whole advantage of the ensemble shrinks.

The sequential bootstrap corrects this: an example’s draw probability falls once an overlapping example has already been drawn. The result is draws with considerably higher average uniqueness, and ensembles whose members actually differ.

The practical core

All three corrections do the same thing: they prevent repetition from being counted as confirmation. That is the same error Level 2 describes for small samples – here formalised and made measurable.

Next

The same overlap also ruins validation. How to repair it is the next article.