Here is the IAQF Trading Historical Backtesting Talk Summary for a quant model and strategy
– Hold-Out Data Testing is insufficient
– 100% Overfitting is where your model will model only the sample space instead of the thing you are trying to model.
– Overfitting is a % of bias in your test data and should be computed as a probability.
– Probability of Backtest Overfitting (PBO) is used to determine how much capital to allocate to each strategy
– Maximum PBO that should be allowed for trading is .20
according to Marcos.
– Overfitting is unlikely with very simple strategies or with complex strategies that use a lot of of data (i.e tick data for HFT) but few variables.
-Neural Nets with 20 or more variables are highly susceptible to overfitting.
– Non-linear estimators like polynomials are more susceptible to noise than linear estimators like OLS
– Symmetric combinations of out of sample test data should
be used for testing (i.e multiple hold-outs)
– Transformations can be used to remove the bias in non-normal distributions and in linear estimators (OLS)
-Neural Nets/Machine Learning can be used to find the optimal transformation needed to remove the bias from the sample space (This is known as Deep Learning)
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2326253
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659
This came from my NYC contact so big thanks to him for sending this
Want to get more stuff like this? Join my FREE newsletter for more info
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!
