Here is the IAQF Trading Historical Backtesting Talk Summary for a quant model and strategy
– Hold-Out Data Testing is insufficient
– 100% Overfitting is where your model will model only the sample space instead of the thing you are trying to model.
– Overfitting is a % of bias in your test data and should be computed as a probability.
– Probability of Backtest Overfitting (PBO) is used to determine how much capital to allocate to each strategy
– Maximum PBO that should be allowed for trading is .20
according to Marcos.
– Overfitting is unlikely with very simple strategies or with complex strategies that use a lot of of data (i.e tick data for HFT) but few variables.
-Neural Nets with 20 or more variables are highly susceptible to overfitting.
– Non-linear estimators like polynomials are more susceptible to noise than linear estimators like OLS
– Symmetric combinations of out of sample test data should
be used for testing (i.e multiple hold-outs)
– Transformations can be used to remove the bias in non-normal distributions and in linear estimators (OLS)
-Neural Nets/Machine Learning can be used to find the optimal transformation needed to remove the bias from the sample space (This is known as Deep Learning)
This came from my NYC contact so big thanks to him for sending this
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!