ADVANCED · ENGLISH · FREE
8. Backtesting and trading statistics
From hypothesis to protocol
Before testing, document the market, period, source, signal rules, time information becomes available, entry, stop, exit, costs and treatment of missing data. Save that version separately. Any change after viewing outcomes creates a new hypothesis.
A backtest applies rules to historical data. A forward test observes new situations after the rules are fixed. Real trading additionally involves infrastructure, actual execution, account constraints and human behaviour. Label these three types of results separately.
Split development and holdout data chronologically. Do not repeatedly use the holdout period to tune settings while calling it independent. When trades overlap, account for dependence between observations and possible information overlap.
Common biases
Look-ahead bias occurs when a decision uses a future low, a swing confirmed later, the final volume of an open candle or a recalculated indicator. Survivorship bias occurs when history contains only assets surviving in today's list. Delisted coins can materially change crypto-market results.
Overfitting occurs when many settings are fitted to noise in a particular history. Testing more variants increases the chance of finding a randomly attractive result. Save every experiment, not just the winner. A smooth region of similar settings is more persuasive than a single sharp peak, but still does not prove future success.
If both stop and target are reached within one OHLC candle, event order is unknown. Use more detailed data or a conservative rule chosen in advance. Do not automatically choose the profitable sequence. When price gaps through the stop, do not record perfect execution at the skipped level.
Costs and executable prices
Include fees on both sides, spread/slippage, funding, rolling dated contracts and minimum-size constraints. Apply rates appropriate to the date and execution role. A constant practice reserve is useful for stress testing, but does not replace observed execution statistics.
A limit entry needs a queue and liquidity model. A simple candle touch can overstate fills. A paper test with unlimited liquidity particularly distorts illiquid instruments and large quantities.
Mini-protocol. Fix the rule; collect data; check gaps and time zones; develop; freeze parameters; test the holdout period; raise costs in a stress scenario; observe new signals; compare discrepancies. A negative result is useful evidence for rejecting a model.
A report that can be checked
A report needs dates, signal and trade counts, unfilled and excluded cases, net result, drawdown, PF, average result in R and calculation method. Show the outcome distribution, not only the final sum. Identify trades whose result depends on unknown event ordering separately.
Exercise. Take 30 consecutive situations without selecting attractive ones. Calculate outcomes using original and doubled hypothetical costs. If the edge disappears, state that limitation. Thirty cases are a process exercise, not a universally sufficient evidential sample.