Backtesting Is Not Proof, It's a First Filter

⏱ 5 min read

Backtesting only earns its keep if you’re honest about what it’s actually testing, and most people aren’t. That’s the whole problem in one line, but let me get to it properly.

Here’s the tension nobody tells you when you start. You backtest because you need to know if an idea has any edge before you risk real money finding out the hard way, that part’s obvious. But the trap is that a backtest is you, alone, with the answer already sitting in the data, deciding after the fact which rules would have worked. I’ve spent ten plus years in engineering where you design against known loads and known failure modes, you run the calculation, the calculation doesn’t change halfway through because you didn’t like the number. Backtesting doesn’t work like that. You’re not calculating against fixed physics, you’re pattern matching against a chart where you already know exactly what happened next, and your brain will quietly nudge every parameter, every entry filter, every stop loss placement until the equity curve looks clean. Nobody does this maliciously. It’s just what a brain does when it can see the future and is told to optimize.

That’s the core difference between backtest and forward test, and it’s not really about time, it’s about whether the outcome was known when the decision was made. A backtest, you already know Tuesday’s candle closed red before you decide your Monday rule should have avoided it. A forward test, you’re making the call on Monday with no idea what Tuesday brings, same as live trading, just without your real capital on the line yet. Think of it like a chef who tastes a dish, likes it, then writes down a “recipe” that happens to perfectly match what they already cooked, versus handing that recipe to someone in a different kitchen with different ingredients and watching if it still comes out right. The first one proves nothing except that the chef can describe what they just did. The second one is the actual test.

So the trap is specific, it has a name, curve fitting, and it’s seductive because a backtest with enough tweakable variables can be made to show almost any result you want. Enough indicators, enough parameter combinations, enough “let me just adjust this one filter,” and you will eventually find a version of your system that returns 40% a year with a beautiful equity curve, on data that’s already happened. That curve is fiction. It’s not an edge, it’s a description, after the fact, of one specific path price happened to take. The moment you deploy it forward, market conditions shift slightly and the “edge” evaporates because it was never an edge to begin with, it was an autopsy dressed up as a diagnosis.

If the trap is real and it genuinely misleads people, the honest question is why bother backtesting at all, why not just forward test everything from day one. And the answer is cost. Forward testing a bad idea takes months to prove wrong, and if that idea has no logical foundation, you’ve burned real time and probably real capital learning what a five minute backtest would have told you for free. Backtesting isn’t there to prove your system works, that’s the part people get backwards. It’s there to quickly kill the ideas that clearly don’t, and to sanity check that your logic isn’t structurally broken before you spend the months of forward testing actually needed to know if it holds up live. Same as running a load calculation before you pour concrete, the calculation doesn’t guarantee the building stands forever under every real world condition it’ll face, but it tells you fast whether the design is fundamentally sound enough to be worth building.

I used to treat a good backtest as validation, as in, the numbers are green, the system works, time to trade it. That’s a wrong mindset. What I’ve landed on is that a backtest only earns the right to move forward into a forward test, it never earns the right to move straight into live capital. So if you’re building a system right now, don’t ask “does my backtest look good.” Ask “would this still look good if I’d locked every rule before seeing this data,” and if you can’t honestly answer yes, put it through a real forward test with paper or minimum size before a single serious dollar goes in.