Tips and tricks to avoid overfitting in trading
On 03 September 2024 - tagged evaluation, machine-learning
In trading, overfitting occurs when your model or backtest shows great performance on historical data but fails on live trading. It is crucial to avoid this problem, because it can cost you a lot of time spent on debugging and tuning bad model and also a lot of money if you actually run the model in production.
We have already covered a few ways to detect and avoid overfitting in the Quant strategy overfitting intro post. Those tricks were however generic for all ML models, not just trading. In this post, we will cover some more advanced tips and tricks to avoid overfitting trading strategies.
Methods #
For high-freq and mid-freq strategies taking hundreds of trades every month while having just a few parameters, it's pretty easy to spot overfitting. Just check the equity curve and if it's relatively straight you're safe. If you see a few spikes up making most of your profits, you're probably overfitting (case #1 plot). You can even automate this by checking or optimizing sharpe ratio instead of absolute return/pnl. In this case, it is relatively safe to not even use out of sample data, although their use is definitely recommended.
Note that this doesn't work for strategies with tens+ parameters, eg. ML models. Those can overfit too perfectly and you need to use out of sample data or walk-forward optimization to avoid this as discussed in the previous post linked above (case #2 plot).
When using out of sample testing, you often run into an equity curve that flattens at the end. That might either mean that your model is overfitting or that you the strategy is becoming less profitable and the market inefficiency you want to trade is being arbitraged away (dead strat plot). You can usually guess which one it is by looking at the equity curve behavior near the out of sample split threshold or moving to the more sophisticated method of walk-forward optimization.
Follow up articles in overfitting series #
- Previous: Algo-trading resources - online
- Next: Analysing overfitting on individual model features
- Back to: Newest article list
New posts are announced on twitter @jan_skoda or @crypto_lake_com, so follow us!