Tag Archives: Neural Networks

Here is an hour long Microsoft Developer Network MSDN video on how to build neural networks with Visual Studio

Here is an hour long Microsoft Developer Network MSDN video on how to build neural networks with Visual Studio

This came in from my NYC contact so thanks for from him. This could easily be one of the best ones I have seen on this topic.

http://channel9.msdn.com/Events/Build/2013/2-401

Learn through my FREE newsletter if this topic will help in building within my trading environment

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Why Genetic Algorithms and Neural Networks will always be a lower priority of forecasting the markets vs quant or even technical analysis

 

 

Why Genetic  Algorithms and Neural Networks will always be a lower priority of forecasting the markets vs quant or even technical analysis

I just got this message from someone on Linked In:

 

 

Bryan I need to know more details about this GPU CUDA 3rd party high level C++ library for math awesomeness with genetic algorith,, please give me more details , I think i have for you a Job implementations .

First off, I appreciate people asking me about jobs to implement this and that. I just turned away about 3 requests like this in the past week. Sorry but we just focus on proven concepts that are part of toolboxes within Matlab.

My opinion on  genetic algorithms and neural network is pretty simple. They take less priority than pure math or statistical methods. I just feel GA and NN are just concepts in trying to train something that is dynamically changing within markets. One of the reasons I like Matlab is it has a highly secretive weapon that I will never release to the public. Even R does not that have this capability. I will tell you that this feature in Matlab, NN is one way to measure how data flows.

Disadvantages:
Certain optimisation problems (they are called variant problems) cannot be solved by means of genetic algorithms. This occurs due to poorly known fitness functions which generate bad chromosome blocks in spite of the fact that only good chromosome blocks cross-over.
There is no absolute assurance that a genetic algorithm will find a global optimum. It happens very often when the populations have a lot of subjects.
Like other artificial intelligence techniques, the genetic algorithm cannot assure constant optimisation response times. Even more, the difference between the shortest and the longest optimisation response time is much larger than with conventional gradient methods. This unfortunate genetic algorithm property limits the genetic algorithms’ use in real time applications.
Genetic algorithm applications in controls which are performed in real time are limited because of random solutions and convergence, in other words this means that the entire population is improving, but this could not be said for an individual within this population. Therefore, it is unreasonable to use genetic algorithms for on-line controls in real systems without testing them first on a simulation model.

http://www.ro.feri.uni-mb.si/predmeti/int_reg/Predavanja/Eng/3.Genetic%20algorithm/_18.html

As for CUDA or GPU, I have already proven to myself that FPGA is still lower latency than CUDA or GPU. It can only be used for backtesing or real time analytics with a delay that would make the use of it kind of useless in a live trading environment. Just look for FPGA on this site for some videos that I have posted about it.

 

 

 

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant development: What types of neural networks are most appropriate for algorithmic trading strategy? From 5-min upto daily time frame

Quant development: What types of neural networks are most appropriate for algorithmic trading strategy? From 5-min upto daily time frame

 

I’ve seen too many neural net programs blow up.

 

It is like asking what brand of hammer is best for nailing a nail. The best type of neural net is the one that you understand and can figure out why it predicts what it predicts. If you don’t… as said… it will eventually “blow up”.

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Neural Networks in Swing Trading

Neural Networks in Swing Trading
Mixing Back propagation NN algorithms with complementary models in trading strategy design. Technical discuss. White papers are welcomed.
—–
First of all, does there really exist NN based trading models?
—-
I’m also interested by feedbacks about neural network in trading strategy. There are a lot of possibilities using NN.How to train the system ? What kind of Inputs ? Output expected…As far as I’m concerned, I tried in the past to use genetic algorithm for the training.I used as input some technical Indicators ( for instance MMA 50, MMA 10 …).I was expecting to find the direction to take, so my output was a double between -1 and 1. if output at a given time is close to 1(-1), It means that I have to take a long(short) position.My results was quite bad (sorry I don’t remember exact figures)… I didn’t try harder because of busy schedule. Now, I think I will take the time to search again 🙂

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Are neural networks and genetic algorithms interesting research fields for financial modelling next time?

Are neural networks and genetic algorithms interesting research fields for financial modelling next time?

Sorry for intruding, I just saw this topic and since I am teaching about both and doing research in finance with genetic algorithms maybe I can say something. About neural networks in general they are good and work when there is nothing known about the process. They are very useful to learn about what happens in black boxes. They do predict and actually it is very easy to obtain predictions given the inputs. When applied to financial data they are not very good even compared with something really simple such as a linear time series. The reason is simple we know more and can include more about the structure of the data then the nn can possibly learn. I however do believe that nn can be used for complex things that are hard to study otherwise. For example, what triggers change in behavior by studying shifts using a large number of assets. The data has to be selected carefully and clearly the nn can only learn about types of shifts already encountered but it should be possible.

Finally, about the genetic algorithms I am using something like that (a particle filtering technique) to detect shifts in volatility. That actually works really well. So yes, I do believe they have a future but one needs to understand how they work and their limitations before applying them.

you might be looking for something like MATLAB with its ANN/Global Optimization toolkit. It’s straightforward, includes several different algorithms and structures, and has built-in GUI tutorials to get you started. MATLAB has a big brand name, and you can switch to command-line code if you like after you get up to speed

NNs were experimented with by many of the big houses during a fashionable period. None of them that I am aware of are still using them. Are there any successful funds out there using NNs?

I am fairly new in comparison to these distinguished gentlemen, but so far I have found GA’s to be most effective for the same reasons as above. However I caution you need a HUGE data sample to test on, and only accept those which generate a significant number of trades with a reasonable equity curve growth to prevent the overfitting. Even then, it’s hard for me to find good systems without having multiple entry and exit rules staggered with the moment of trigger of each rule, and each type of trigger varying the next rule to tip an entry or ramp-up or change type of exit depending on behaviour, which gets very time-consuming computationally, as well, there may be some research there on how to cut down the search space for the time problem, good luck

From a Linked In Group

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Fascinating discussion on results of Neural Networks in world of quant part 2

Fascinating discussion on results of Neural Networks in world of quant part 2
We have heard about Nerual Networks in world of quant. The following was found in a Linked In discussion:
OK. Let’s start from the beginning for there is too many things going on. First, we need to filter the market. I’ll use a method of mine call the max-min filter. Look for a formation where the next high is lower than the current high. Make sure that highs before the current high are all at most equal or you find a lower high. Then declare the current high a market maximum. Do the same on an inverted pattern to find the current minimum. Now you have a market with minima and maxima. Make sure they alternate. If you find two consecutive max (or min) look for the lowest low (highest high) between them and mark as min (max). Now the alternate perfectly. By connecting the dots in time you have a perfect characterization of the market. Remember that you must be certain what happen first in each bar the high or the low.

With this set of points we can then characterize a trade by saying within x ticks of a maximum I want to short this market or, within 25% of the previous down leg, I want to short the current leg. Now we have out perfect objective market.

Take each trade from the series and scale. This is done like this, take the close of the market bar at entrypoint and call it zero. Use that value to subtract from all your embedding (OHLC of previous bars), and do the same for you objective, say, 80% of the current leg.
Notice that all you have now is a bunch of unrelated trades with an embedding and a target. And that is all you have on a ANN because they are memoryless systems. I designed ANN with short term memory but never applied to the market.
IEEE Transactions on Neural Networks, volume me 6 (1995) Tom, M.D. and M.F. Tenorio, A neural computation model with short term memory, pp. 387-397. Thathachar, M.A.L. and V.V. Phansalkar, Learning the global …
www.ph.tn.tudelft.nl/…/IEEE.Transactions.on.Neural.Networks.6.html – Em cache –

So back to our problem, we could do straight training, no harm done because ANNs do not car what came first. Except when you use training. Training algorithms in their majority are gradient based. A person training 1988-2000 will get a different result that 1989-2001. That is absurd. But neural network follow same of the same learning done in the brain, and therefore you can make it easy for learning or not.
Tenorio*, M. F., R. G. Schwartz, and M. D. Tom, “Adaptive Network as a Model for Human Speech Development,” Computers in Human Behavior, Pergamon Press, vol. 6, no. 4, pp. 291-313, 1990.

So let’s make it each on our ANN that already is doing a tremendous task. Let’s decrease the discontinuities and the high frequencies and the total bandwidth of the signal (less hidden units) by using a trick we use with children. First the basics then teach it the exception. Cluster all the trades by hamming distance of the multidimensional input point. Sort them by closeness. Then take the sort set and resort them by closeness in output difference. Now all situations that are similar are together. Some thing like:
Who loves you? Daddy.
Who things you are the greatest? Daddy
Who is king? Daddy
Who…? Daddy
You get my drift.

Now we are ready to train. The choice of ANN should align with the task at hand. This is a clustering problem and a RFB ANN will do much better than a sigmoidal one. Perform the training. Now we need an exit strategy. Measure by eye how close to the actual filtered result the ANN is signaling entry, and how many ticks does it indicate for the move. If you are satisfied this is a good strategy, else add trailing stop and the works. Test is on the training set and do a post mortem analysis (in the book I have indicated previously) to see how well you are performing during the trade. Fix it accordingly and try the test set. A couple more adjustments and you are done.

The main thing to take from here is that it is not a blind exercise of raw market learning, neither such things exist. The neural network is always performing a classification of patterns when you use bars as inputs.

Now to the money management scheme. The same way that I can take ANY input signal and make an excellent or a poor systems (discussed in a posting in another discussion in this group) I can also take ANY trading system and make it glorious. It all depends on adjust the exits and then use a non fixed number of contracts. To see if this works, get the set of trade results and place them in the filtered market. Now in that order plot past trade x current trade final equity. You want all the points in the first and 3 quadrant. Histogram the positive trades x past trades, and negative trades x past trades. You should see serial correlations, when you win, you will win for a while, and when you lose it is the same. Transitions between winning and losing are rare. So now if you have streaks of win and lost plan you traders accordingly. This will make a 55% system into a 90% system with low drawdown and very high return.

If you follow the recipes here you should be in good shape for an excellent trade systems. There are many other variants on this specially when dealing with entire portfolio balance.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How any quant time series can be profitable through neural networks?

How any quant time series can be profitable through neural networks?
This was a discussion found on Linked In:
These results are in a single instrument, single series, and it is totally based on money management. It was done to prove a simple point: any series can be profitable with a very tiny edge. Now adding risk and portfolio things can get a lot better. Send me a series and I can show you the equity curve without predictors.
I have been doing research on predictors since the late 80’s and on trade systems since the early 90’s, We did a competition mentioned earlier on this tread using the best researchers in both prediction and trading tasks. No one did well.

When I hear that someone is doing NN or any other technique with success over 75% I get really concerned. What does the percentage means? entries? exit? correct close trades? None of this has anything to do with the NN or any other technique that you are using. The predictor is now deep into a system. A measure of prediction accuracy is the normalized root mean squared error, not percentage. For trade systems I would like to see maximum drawdown, equity per period, total return over investment and modified Sharpe ratio.

If someone wants to tell me that they are doing well, send me a predicted curve overlaid by a market curve in and around a few market turns. It the delay in predicting the turn is equal to n/2 (number of time steps used in the prediction) then you are no better that a moving average.

Most of what people call successful prediction is the trend effect of the market. During trends they are right. The markets are NOT a random process and generate a perfect bell curve as academics would have us believe. It is a bell with elongated tails by the trends. This causes people to fool themselves in that their predictor works.

On the long run, the prediction will experience severe drawdowns and eventually drift out of tune (the market change as said above).

In order to predict the market, one has to think of it not a using a magical bullet of some statistical or machine intelligence method to discover its secret. The market is more like a swarm behavior. We see swarms in Nature all the time. Not one species is equal to another. There are several outside strong forces acting on the markets at different time scales and large in market agents as well that makes it impossible to create a close form prediction. It does not matter your horizon or what you are trading.

1. Learn that prediction is not the key.
2. Try using tracking instead of prediction.
3. Try to use pattern classifiers instead of tracking.
4. Add to it a trade classifier (Is this trade going to be successful)
5. Concentrate your power on the exit strategy. (I can make or lose money on any entry).
6. Think of it as a game (game theory). No matter how your system works, turn it into profitable with a post mortem analysis.
7. yes, then add money management, risk and portfolio later.

You will make money on the market with just about anything. but you will not make a good predictor. if you make one, please share with me the results as I have spent much time search for it.

Yes, this approach is sweet for options and multioption strategies.

If you are having a hard time with your system or would like to improve it, all I need is a market curve with your open equity curve over it.

Go and be profitable!

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Are neural networks the next way for HFT, hedge funds, and quants to predict the markets?

Are neural networks the next way for HFT, hedge funds, and quants to predict the markets?
I am agree with you, predictive analysis is a powerful decision-aid for high frequency traders which trade each 30 minutes or each 60 minutes ;).

I also think this prediction time interval is ideal. Are you agree?

This was a discussion found on Linked In:
Predictive analysis (which include neural networks techniques) helps manager to make better timing (enter or exit decision) for example.

Look at the INFORMS Data Mining Contest ( http://kaggle.com/informs2010 ). This contest requires participants to develop a model that predicts stock price movements (over 60 minutes) at five minute intervals (provided with intraday trading data showing stock price movements at five minute intervals, sectoral data, economic data, experts’ predictions and indices).

A lot of competitor using neural network and others predictive analysis techniques to correctly predict what will happen in 60 minutes.

In addition, my company develops this kind of solution since 10 year and it works ;).

So, I agree with you, the use of neural network will increase.
Our company developed in the period 1996-2002 based upon Fuzzy Logics of
Professor Zadeh not a stock-predicting system however a company predicting system.

Since 2003 this is real life on the Dutch Stock Exchange. Whereas the index developed 3% the predicted companies achieved 130% yield. The predicting power of bankrupty increased to 97%. In the CAP-profile now a 99,9% track-record of the ideal Lorentz-curve can be established. Notable and documented results are : Enron, Worldcom, Parmalat, Ahold, and many local ones.
The reason so few advanced methods are used in HTF is the basic contraidiction between speed and depth. HFT trading often uses special operating systems stripped of all non essential code so it will run just a littlel faster. The more processing done the slower your response in HFT. In doing analytics models for non-HFT one can run as much as one wants to get a good trade…if running 10 ot 20 seperate models takes an extra 3 seconds to get a profitable trade, I really don’t car as I’m trading 30 or 60 min. basr. However this delay would kill any HFT strategy.
he ideal prediction time window is a function of cost to trade and average range at a given time granularity. I’ve found approximately equal predictability in a time range of 1 min to 60 minutes, at least in major Forex pairs.

However at shorter times windows there is of course less range in prices. So a particular pair might have average range of 3 Pips for 1 minute bars and 20 or 30 Pips for 1 hour bars. If cost of trading is 2 Pips then in trading 1 minute bars 66% of the average range is lost to this cost. In trading 1 hour bars the loss is only 10%.

So one needs to consider this along with the risk of time in open positions in selecting the best time granularity on which to base predictive trading systems
I don’t trade stocks, just Futures and Forex. The closest I get is modeling the DJIA and similar indexs for use in trading futures based on them. Stocks have a lot of noise in them… a rumor of dubious validity appears on the Internet about a particular companies legal issues on a patent, the CEO gets caught having serious personal issues (smoking pot at bridge tournaments while his firm falls toward insolvency) and the price is distorted. \\

In contrast Futures and Forex are often much more liquid (able to absorb noise) and not subject as much to “hot tips” on a company’s products, legal issues or the moral and personality problems of the top executives.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Are neural networks best at predicting markets vs quant algos?

Are neural networks best at predicting markets vs quant algos?
This was a discussion found on Linked In:
Just Google the terms “neural network, finincial time series, prediction, trading PDF” and you will find a number of academic papers documenting that while not perfect their performance is far better than human traders, expert systems or collections of algos generated by eager young Quants on Wall Street.

The out of sample failure is very common and is caused by people using very complex and sophisticated technology without the required skills or training. I won’t bore the discussing with how this is done or the methods of avoiding what you describe which are well know and documented by those with both skills and experience in working with NNs.
May I also say that the inadequate results from NNs are not due to the inadequate training of the user of a NN. They are due to the fact that financial signals are highly nonstationary, and hence the training of neural networks has to be correspondingly responsive to this environment. However, while there are techniques/algorithms for “adaptive” training of neural networks (and I have published some), the number of “samples”in a highly volitile and non-stationary environment is not sufficient to provide adequate tracking of the data and hence the NN are trained incorrectly i.e. one is always very “fuzzily” training the NNs.

Thus while I do not wish to run down NNs for prediction, my own personal and long standing experience stemming from deep algorithmic research does not support the use of NNs in a highly non-stationary environment. As far as the publications on this matter are concerned I think you will find from a very careful study that they are by and large either “in sample” results or that the environment in which they have been applied is not volatile enough (ie the underlying market behaviour is not changing fast enough).

In fact I can now prove that there exist a set of conditions that must be satisfied a priori by a set of data for it to be predictable by a NN. This work will be published in the near future
y experience is using NNs as drivers for short term trading systems in futures and Forex is very different than your experience: they can provide the basis for stable and profitable trading systems.

This of course might be to different approaches and goals. I have no need to forecast each moment of market activity. Markets have a chaotic component and when in that mode prediction is of course impossible. When not chaotic there are complex topologies that an NN can learn through training that result in predictable outcomes often enough to generate profitable trades.

I’m not alone in this discovery, as on LinkedIn, in related forums and topics you will find other having similar results as well.

To your points:

A) You said: “), the number of “samples”in a highly volitile and non-stationary environment is not sufficient to provide adequate tracking of the data and hence the NN are trained incorrectly i.e. one is always very “fuzzily” training the NNs.”

What do you view as a sufficient number of samples? For short term trading, say 5 minute bars in Forex, there are about 150,000 bars of data generated per year. Going back 7 years one has over 1,000,000 data bars.

From my own work I’ve found one does need perhaps 30,000 to 50,000 bars for training at this time granularity.

B) You said: “As far as the publications on this matter are concerned I think you will find from a very careful study that they are by and large either “in sample” results or that the environment in which they have been applied is not volatile enough (ie the underlying market behaviour is not changing fast enough).”
So if I understand your meaning in the last phrase: an NN only appears to work because it has not yet encountered a market changing fast enough to cause it to fail.

If your requirement is that it perform with 100% accuracy in all conditions including new ones it has not been exposed to in training then of course you are correct. Form an academic sense this might be important or in a corporate stetting where the bank has to manage on going and constant positions in multiple markets/instruments.

However, for more general trading it is largely irrelevant. If there is a prediction based on conditions the trained NN has learned, you trade. When the market is in a state that is “changing too fast”, you are flat and not trading. This is accomplished though various means: parliamentary models, where dozens of models vote on what to do, failure models, where an NN is trained to predict the failure in prediction by a base model or parliament of models.

Some examples that undermine your assertion in the first phrase in B) above:

http://www.smartquant.com/references/NeuralNetworks/neural31.pdf

http://www.tristanfletcher.co.uk/FX%20Carry%20Paper.pdf

Quoting from the above:

“4 Conclusions
Assuming conservative estimates of trading costs, over
the 10.5 year (2751 trading day) rolling out of sample pe-
riod investigated, improvements of 120% in MAR ratio,
* in Sortino and 80% in Sharpe relative to the `Al-

ways In’ benchmark were found. Furthermore, the extent
of the maximum draw-down was reduced by 19% and the
longest draw-down period was 53% shorter.”

Like I said a little Googling is always a good idea.

However do not expect to find an academic paper detailing all that is needed to do this successfully. For obvious reasons, academics who get to that point tend to stop publishing just before leaving the university to start hedge funds.
He did not design a predictor or entry signal but a moving, adaptive tracker. This is a much more robust design that a simple predictor that needs stability for a long time. The market tends to behave predictably with chaotic intervals in between structure intervals. So a tracker requires less of the system.

I need to say that maintaing this discussion on ANN is really a disservice to what is know in the literature and to a number of other algorithms. I feel a designer should have a good grasp of the algorithms and the design he is trying to achieve.

There are algorithms for when the inputs are unknown, for when the dimensionality is high, for small training sets, for predictors, classifiers and trackers. ANN just got more advertising outside the statistical community, hence the interest. It is good for some things and it requires some expert tweaking. Stability can be a problem as well as optimization of the weights but there are solutions to all that. The architecture of the problem is of great importance as well: how many machines to use, how to deal with off-sample, input variable design and how much you are asking the machines to do.

At the end, since this focuses on a trading system design discussion I would like to leave two thoughts:
1. The entry signal has very little value as compared to the exit signal.
2. Money management is everything.

I can show you how to make money on a truly random variable on 55% wining system consistently.

So, design with the objective in mind, be light on the requirements (do not require to pick top and bottoms) and choose the right algorithms for the problem.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

What are the best tools to use for quant advanced neural networks?

What are the best tools to use for quant advanced neural networks?
This was a discussion found on Linked In:
I use neuroshell by ward systems. I don’t implement it’s trading signals religiously, though. I do use it as the basis for my trades and reccommendations athttp://www.cotsignals.com . You can see some examples of how I use it. I combine a neural network trigger with commitment of traders data.
have been using feedforward backpropagation neural network with partial recurrency. The latter facilitates the convergence at the expense of “rocky” error surface. I find ANN’s as a fantastic non-linear regression tool. However, the secret is not really in the regression tool but, obviously, the inputs you are feeding into it. I use jitter and a number of other techniques to enforce generalisation (i.e. prevent curve fitting). I use certain decomposition techniques on the input distribution of returns and look at thick tails, apart from other things, to detect regime shifts. Overall, I am a big fan of ANN’s, however, the majority of the models based on the I have seen so far fail miserably which makes perfect sense. My advice would be to take the majority of the models with a grain of salt, respect and understand leveraged trend-followers and technical traders, because if you get your model(s) right, they will be funding your R&D and perhaps a bit more! 🙂 One other thing to bear in mind is that human brain has the luxury of 10^11 degrees of freedom — the trick is to use at least some of them to co-operate with much simpler ANN-based models!

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!