Are neural networks best at predicting markets vs quant algos?
This was a discussion found on Linked In:
Just Google the terms “neural network, finincial time series, prediction, trading PDF” and you will find a number of academic papers documenting that while not perfect their performance is far better than human traders, expert systems or collections of algos generated by eager young Quants on Wall Street.
The out of sample failure is very common and is caused by people using very complex and sophisticated technology without the required skills or training. I won’t bore the discussing with how this is done or the methods of avoiding what you describe which are well know and documented by those with both skills and experience in working with NNs.
May I also say that the inadequate results from NNs are not due to the inadequate training of the user of a NN. They are due to the fact that financial signals are highly nonstationary, and hence the training of neural networks has to be correspondingly responsive to this environment. However, while there are techniques/algorithms for “adaptive” training of neural networks (and I have published some), the number of “samples”in a highly volitile and non-stationary environment is not sufficient to provide adequate tracking of the data and hence the NN are trained incorrectly i.e. one is always very “fuzzily” training the NNs.
Thus while I do not wish to run down NNs for prediction, my own personal and long standing experience stemming from deep algorithmic research does not support the use of NNs in a highly non-stationary environment. As far as the publications on this matter are concerned I think you will find from a very careful study that they are by and large either “in sample” results or that the environment in which they have been applied is not volatile enough (ie the underlying market behaviour is not changing fast enough).
In fact I can now prove that there exist a set of conditions that must be satisfied a priori by a set of data for it to be predictable by a NN. This work will be published in the near future
y experience is using NNs as drivers for short term trading systems in futures and Forex is very different than your experience: they can provide the basis for stable and profitable trading systems.
This of course might be to different approaches and goals. I have no need to forecast each moment of market activity. Markets have a chaotic component and when in that mode prediction is of course impossible. When not chaotic there are complex topologies that an NN can learn through training that result in predictable outcomes often enough to generate profitable trades.
I’m not alone in this discovery, as on LinkedIn, in related forums and topics you will find other having similar results as well.
To your points:
A) You said: “), the number of “samples”in a highly volitile and non-stationary environment is not sufficient to provide adequate tracking of the data and hence the NN are trained incorrectly i.e. one is always very “fuzzily” training the NNs.”
What do you view as a sufficient number of samples? For short term trading, say 5 minute bars in Forex, there are about 150,000 bars of data generated per year. Going back 7 years one has over 1,000,000 data bars.
From my own work I’ve found one does need perhaps 30,000 to 50,000 bars for training at this time granularity.
B) You said: “As far as the publications on this matter are concerned I think you will find from a very careful study that they are by and large either “in sample” results or that the environment in which they have been applied is not volatile enough (ie the underlying market behaviour is not changing fast enough).”
So if I understand your meaning in the last phrase: an NN only appears to work because it has not yet encountered a market changing fast enough to cause it to fail.
If your requirement is that it perform with 100% accuracy in all conditions including new ones it has not been exposed to in training then of course you are correct. Form an academic sense this might be important or in a corporate stetting where the bank has to manage on going and constant positions in multiple markets/instruments.
However, for more general trading it is largely irrelevant. If there is a prediction based on conditions the trained NN has learned, you trade. When the market is in a state that is “changing too fast”, you are flat and not trading. This is accomplished though various means: parliamentary models, where dozens of models vote on what to do, failure models, where an NN is trained to predict the failure in prediction by a base model or parliament of models.
Some examples that undermine your assertion in the first phrase in B) above:
Quoting from the above:
Assuming conservative estimates of trading costs, over
the 10.5 year (2751 trading day) rolling out of sample pe-
riod investigated, improvements of 120% in MAR ratio,
* in Sortino and 80% in Sharpe relative to the `Al-
ways In’ benchmark were found. Furthermore, the extent
of the maximum draw-down was reduced by 19% and the
longest draw-down period was 53% shorter.”
Like I said a little Googling is always a good idea.
However do not expect to find an academic paper detailing all that is needed to do this successfully. For obvious reasons, academics who get to that point tend to stop publishing just before leaving the university to start hedge funds.
He did not design a predictor or entry signal but a moving, adaptive tracker. This is a much more robust design that a simple predictor that needs stability for a long time. The market tends to behave predictably with chaotic intervals in between structure intervals. So a tracker requires less of the system.
I need to say that maintaing this discussion on ANN is really a disservice to what is know in the literature and to a number of other algorithms. I feel a designer should have a good grasp of the algorithms and the design he is trying to achieve.
There are algorithms for when the inputs are unknown, for when the dimensionality is high, for small training sets, for predictors, classifiers and trackers. ANN just got more advertising outside the statistical community, hence the interest. It is good for some things and it requires some expert tweaking. Stability can be a problem as well as optimization of the weights but there are solutions to all that. The architecture of the problem is of great importance as well: how many machines to use, how to deal with off-sample, input variable design and how much you are asking the machines to do.
At the end, since this focuses on a trading system design discussion I would like to leave two thoughts:
1. The entry signal has very little value as compared to the exit signal.
2. Money management is everything.
I can show you how to make money on a truly random variable on 55% wining system consistently.
So, design with the objective in mind, be light on the requirements (do not require to pick top and bottoms) and choose the right algorithms for the problem.NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!