{"format":"alysly-page/1","url":"https://alysly.com/learn/cross-validation/","json":"https://alysly.com/json/learn/cross-validation/index.json","lang":"en-US","title":"Purged Cross-Validation and Embargo for Financial Data","description":"Why k-fold cross-validation fails on price data and how purging, embargo and combinatorial methods close the leak.","published":"2026-08-02T15:54:51+02:00","modified":"2026-11-01T18:26:04+01:00","alternates":[{"lang":"de-DE","url":"https://alysly.com/de/lernen/kreuzvalidierung/","json":"https://alysly.com/json/de/lernen/kreuzvalidierung/index.json"}],"about":"Machine-readable version of the page at url. It contains the text, tables and key figures published there, but not the interactive charts and tools.","breadcrumb":[{"name":"Investing & Trading Basics","url":"https://alysly.com/learn/"},{"name":"Level 3","url":"https://alysly.com/learn/#level-3"},{"name":"Cross-Validation Without Leakage","url":"https://alysly.com/learn/cross-validation/"}],"content":[{"type":"paragraph","text":"Level 3 AI in Trading · Part 7/9"},{"type":"heading","level":1,"text":"Cross-Validation Without Leakage"},{"type":"paragraph","text":"By Alexander Bock, founder and data analyst","links":[{"text":"Alexander Bock","href":"https://alysly.com/about/"}]},{"type":"paragraph","text":"Cross-validation is the standard tool for checking whether a model generalizes. On financial data, in its usual form, it systematically returns results that are too good."},{"type":"heading","level":2,"id":"why-k-fold-leaks","text":"Why K-Fold Leaks"},{"type":"paragraph","text":"Standard k-fold cross-validation splits the data randomly into k blocks, trains on k−1 of them and tests on the rest. It assumes independence."},{"type":"paragraph","text":"On price data, that creates two leaks:"},{"type":"paragraph","text":"Through labels. A training example shortly before the test block has a label that reaches into the test block. Its outcome is driven by the same price moves as the test examples. The model has already seen part of the answer."},{"type":"paragraph","text":"Through autocorrelation. Prices at neighboring points in time are similar. A model that knows Monday’s and Wednesday’s prices can almost guess Tuesday’s without having learned anything economic."},{"type":"paragraph","text":"The result is always the same: excellent validation scores and a disappointing live run."},{"type":"heading","level":2,"id":"purging-and-embargo","text":"Purging and Embargo"},{"type":"image","src":"https://alysly.com/img/ta/purged-cv.6aeb8b8f2fd96d61cdd9083d3a93613f12540600c26f13270bcf33c07f7ee8b1.webp","alt":"Timeline with training blocks, a test block, purge zones and an embargo"},{"type":"paragraph","text":"With overlapping labels, information leaks across the boundary between training and test. (1) Training data, (2) test window. (3) The purge zone removes training examples whose labels extend into the test window. (4) The embargo additionally blocks a short span right after the test, because price series are serially correlated. Without both, cross-validation looks great and live trading disappoints — the model simply knew part of the answer."},{"type":"paragraph","text":"Purging removes from the training set every example whose label window overlaps the test window. That closes the leak through labels."},{"type":"paragraph","text":"Embargo additionally blocks a short span directly after the test window. The reason is autocorrelation: examples right after the test still contain information about its end, even when their label windows do not formally overlap. A common size is one to five percent of the length of the data."},{"type":"paragraph","text":"Together they cost training data – with ten blocks, easily a mid single-digit percentage. That loss is the price of having a metric that means anything at all."},{"type":"heading","level":2,"id":"why-a-single-test-period-is-not-enough","text":"Why a Single Test Period Is Not Enough"},{"type":"paragraph","text":"Even with purging, one problem remains: a single split gives you exactly one number. You cannot see from it whether the number is stable or whether it depends on a lucky cut point."},{"type":"paragraph","text":"Combinatorial purged cross-validation deals with this. Instead of one test block, several blocks are used as test sets in different combinations. That produces many different train/test paths, and therefore a distribution of results instead of a single figure."},{"type":"paragraph","text":"The benefit is the same as with the Monte Carlo tool for portfolios. You see the range of outcomes the same data could just as easily have produced. A strategy whose Sharpe ratio swings between 0.2 and 1.8 depending on the split is very different from one that stays steady at 0.9. That holds even when both average out to the same value.","links":[{"text":"Monte Carlo tool","href":"https://alysly.com/tools/monte-carlo/"}]},{"type":"heading","level":2,"id":"what-remains","text":"What Remains"},{"type":"paragraph","text":"Even a cleanly purged procedure only answers whether the model generalizes on this data. It does not answer whether the relationship still exists tomorrow, and it does not correct for the number of variants you tried."},{"type":"paragraph","text":"That second question is the last and largest error – and the subject of the next article."},{"type":"heading","level":2,"id":"next","text":"Next"},{"type":"paragraph","text":"How many attempts are hiding inside a good backtest?"},{"type":"list","ordered":false,"items":[{"text":"Previous Overlapping Examples and Sample Weights","href":"https://alysly.com/learn/sample-weights/"},{"text":"Next Backtest Overfitting: The Number of Trials Counts","href":"https://alysly.com/learn/backtest-overfitting/"}]},{"type":"heading","level":2,"text":"Indices"},{"type":"table","caption":"Indices, performance over the selected period","columns":["Index","1M"],"rows":[[{"text":"Dow Jones","href":"https://alysly.com/markets/indices/dow-jones/"},{"text":"+1.0%","data":{"m1":"+1.0%","m3":"-4.3%","m6":"+4.9%","ytd":"+7.6%","max":"+30.0%"}}],[{"text":"S&P 500","href":"https://alysly.com/markets/indices/sp500/"},{"text":"-2.6%","data":{"m1":"-2.6%","m3":"-2.2%","m6":"+4.2%","ytd":"+10.6%","max":"+35.6%"}}],[{"text":"MSCI World","href":"https://alysly.com/markets/indices/msci-world/"},{"text":"-2.0%","data":{"m1":"-2.0%","m3":"-2.4%","m6":"+3.7%","ytd":"+10.0%","max":"+35.0%"}}],[{"text":"MSCI ACWI","href":"https://alysly.com/markets/indices/msci-acwi/"},{"text":"-3.3%","data":{"m1":"-3.3%","m3":"-3.0%","m6":"+2.3%","ytd":"+10.2%","max":"+41.5%"}}]]},{"type":"heading","level":2,"text":"ETFs"},{"type":"table","caption":"ETFs, performance over the selected period","columns":["Fund","1M"],"rows":[[{"text":"SPDR S&P 500 ETF Trust SPY","links":[{"text":"SPDR S&P 500 ETF Trust","href":"https://alysly.com/markets/etfs/us/spy/"}]},{"text":"-2.6%","data":{"m1":"-2.6%","m3":"-2.2%","m6":"+4.2%","ytd":"+10.6%","max":"+35.6%"}}],[{"text":"Vanguard S&P 500 ETF VOO","links":[{"text":"Vanguard S&P 500 ETF","href":"https://alysly.com/markets/etfs/us/voo/"}]},{"text":"-2.6%","data":{"m1":"-2.6%","m3":"-2.2%","m6":"+4.2%","ytd":"+10.6%","max":"+34.1%"}}],[{"text":"iShares Core MSCI World UCITS ETF USD (Acc) EUNL","links":[{"text":"iShares Core MSCI World UCITS ETF USD (Acc)","href":"https://alysly.com/markets/etfs/ishares-core-msci-world-ucits-etf-usd-acc/"}]},{"text":"-2.0%","data":{"m1":"-2.0%","m3":"-2.3%","m6":"+4.3%","ytd":"+10.6%","max":"+487.5%"}}],[{"text":"Xtrackers MSCI Emerging Markets UCITS ETF 1c XMME","links":[{"text":"Xtrackers MSCI Emerging Markets UCITS ETF 1c","href":"https://alysly.com/markets/etfs/xtrackers-msci-emerging-markets-ucits-etf-1c/"}]},{"text":"-9.4%","data":{"m1":"-9.4%","m3":"-4.1%","m6":"-3.3%","ytd":"+14.0%","max":"+93.0%"}}]]},{"type":"heading","level":2,"text":"Stocks"},{"type":"table","caption":"Stocks, performance over the selected period","columns":["Stock","1M"],"rows":[[{"text":"NVIDIA Corporation NVDA","links":[{"text":"NVIDIA Corporation","href":"https://alysly.com/markets/stocks/nvidia-corporation/"},{"text":"United States","href":"https://alysly.com/markets/stocks/browse/?country=United%20States"}]},{"text":"-13.5%","data":{"m1":"-13.5%","m3":"-2.5%","m6":"+5.2%","ytd":"+10.8%","max":"+62.2%"}}],[{"text":"Apple Inc. AAPL","links":[{"text":"Apple Inc.","href":"https://alysly.com/markets/stocks/apple-inc/"},{"text":"United States","href":"https://alysly.com/markets/stocks/browse/?country=United%20States"}]},{"text":"-0.3%","data":{"m1":"-0.3%","m3":"+7.3%","m6":"+16.8%","ytd":"+22.1%","max":"+45.8%"}}],[{"text":"Tesla Inc. TSLA","links":[{"text":"Tesla Inc.","href":"https://alysly.com/markets/stocks/tesla-inc/"},{"text":"United States","href":"https://alysly.com/markets/stocks/browse/?country=United%20States"}]},{"text":"+5.2%","data":{"m1":"+5.2%","m3":"+21.7%","m6":"+2.3%","ytd":"-11.4%","max":"+65.3%"}}]]},{"type":"paragraph","text":"Data note 1","href":"#dn-1"},{"type":"heading","level":2,"text":"More Studies"},{"type":"list","ordered":false,"items":[{"text":"Jul 21, 2026 Collectibles ×5.4 for the same cards, matched-model index, Mar 2022 – Aug 2026 Is the Pokémon Card Market in a Bubble? 4,159 Sales Analyzed Prices for PSA-graded cards multiplied. Measured on the same cards they rose 438%, while the raw median shows 786% because pricier cards came up for sale later.","href":"https://alysly.com/analysis/pokemon-card-market-bubble/"},{"text":"May 19, 2026 Crypto 45.6% Bitcoin’s annualized volatility, Jul 2024 – Sep 2026 Bitcoin in Investment Portfolios: The Real Numbers Bitcoin is calmer than it was and still about three times as volatile as the stock market. It moves with stocks, not with gold.","href":"https://alysly.com/analysis/bitcoin-investment-portfolios/"},{"text":"Mar 17, 2026 Wealth €1.2m needed for €3,000 a month indefinitely at a 3% real return Living Off Your Returns: What It Really Takes The capital you need is arithmetic. The return you assume is a guess, and it can triple the answer.","href":"https://alysly.com/analysis/living-off-returns/"}]},{"type":"paragraph","text":"View all studies","href":"https://alysly.com/analysis/#studies"},{"type":"heading","level":2,"text":"Learn"},{"type":"list","ordered":false,"items":[{"text":"Backtest Overfitting: The Number of Trials Counts The best of 50 trials always looks good – how to correct for it.","href":"https://alysly.com/learn/backtest-overfitting/"},{"text":"Feature Importance and Bet Sizing Which features actually carry the model – and how that becomes a position size.","href":"https://alysly.com/learn/feature-importance/"}]},{"type":"paragraph","text":"Course overview","href":"https://alysly.com/learn/"},{"type":"paragraph","text":"Updated Nov 1, 2026 · Content and figures checked Oct 19, 2026"},{"type":"heading","level":2,"id":"note-legend-h","text":"Data Notes"},{"type":"paragraph","text":"All calculations run automatically and are not investment advice."},{"type":"list","ordered":true,"items":[{"id":"dn-1","text":"Returns computed automatically from the price series, excluding dividends, taxes and costs."}]},{"type":"paragraph","text":"Data and methodology: Alysly · Responsible for content: Alexander Bock","links":[{"text":"Alysly","href":"https://alysly.com/methodology/"},{"text":"Alexander Bock","href":"https://alysly.com/about/"}]},{"type":"paragraph","text":"View all data notes","href":"https://alysly.com/notes/"}],"structured_data":[{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://alysly.com/#org","name":"Alysly","url":"https://alysly.com/","logo":{"@type":"ImageObject","url":"https://alysly.com/img/icon-512.png","width":512,"height":512},"description":"Data-driven insights, transparent analysis and powerful tools for long-term investors.","email":"mail@alysly.com","contactPoint":{"@type":"ContactPoint","contactType":"customer support","email":"mail@alysly.com","availableLanguage":["de","en"]},"address":{"@type":"PostalAddress","addressLocality":"Haltern am See","postalCode":"45721","addressCountry":"DE"},"founder":{"@id":"https://alysly.com/#founder"},"sameAs":["https://www.linkedin.com/company/alyslycom/","https://www.youtube.com/@alyslycom","https://x.com/alyslycom"],"publishingPrinciples":"https://alysly.com/methodology/"},{"@type":"Person","@id":"https://alysly.com/#founder","name":"Alexander Bock","jobTitle":"Founder and data analyst","url":"https://alysly.com/about/","sameAs":["https://www.linkedin.com/in/abxk/"]},{"@type":"WebSite","@id":"https://alysly.com/#site","url":"https://alysly.com/","name":"Alysly","alternateName":"Alysly Investment Analysis","inLanguage":"en-US","publisher":{"@id":"https://alysly.com/#org"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://alysly.com/markets/stocks/browse/?q={search_term_string}"},"query-input":"required name=search_term_string"}},{"@type":"WebPage","@id":"https://alysly.com/learn/cross-validation/#webpage","url":"https://alysly.com/learn/cross-validation/","name":"Cross-Validation Without Leakage – Alysly","description":"Why k-fold cross-validation fails on price data and how purging, embargo and combinatorial methods close the leak.","inLanguage":"en-US","isPartOf":{"@id":"https://alysly.com/#site"},"dateModified":"2026-11-01T18:26:04+01:00","publisher":{"@id":"https://alysly.com/#org"},"mainEntity":{"@id":"https://alysly.com/learn/cross-validation/#entity"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://alysly.com/og.png","width":1200,"height":630}}]},{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Investing & Trading Basics","item":"https://alysly.com/learn/"},{"@type":"ListItem","position":2,"name":"Cross-Validation Without Leakage"}]},{"@context":"https://schema.org","@type":"Article","@id":"https://alysly.com/learn/cross-validation/#entity","headline":"Cross-Validation Without Leakage","description":"Why k-fold cross-validation fails on price data and how purging, embargo and combinatorial methods close the leak.","inLanguage":"en-US","mainEntityOfPage":"https://alysly.com/learn/cross-validation/","image":[{"@type":"ImageObject","height":420,"url":"https://alysly.com/img/ta/purged-cv.6aeb8b8f2fd96d61cdd9083d3a93613f12540600c26f13270bcf33c07f7ee8b1.webp","width":1280}],"isPartOf":{"@type":"CreativeWorkSeries","name":"Investing & Trading Basics","url":"https://alysly.com/learn/"},"educationalLevel":"Level 3","author":{"@id":"https://alysly.com/#founder"},"publisher":{"@id":"https://alysly.com/#org"},"datePublished":"2026-08-02T15:54:51+02:00","dateModified":"2026-11-01T18:26:04+01:00"}],"notice":"© 2026 Alysly.com. Not investment advice. All data without guarantee. Financial information is analyzed and presented partly with AI assistance."}