hat is a very good and valid question, first let me make one thing clear – the “random data” testing does not replace in any way the “traditional” back / forward testing on real historical data!
As for your question, the random data is similar to every-tick data, therefore there is no loss of any information.
For example, let’s say we want to simulate a Forex instrument – we can define it like this:
Starting value – 1.4321
Minimum movement – 0.0001
Now the random data program will start creating millions of options for it, for example – a very basic one can be “in each movement – move randomly 50%-50% up/down by 1 Minimum movement(0.0001)”.
This of course will not be a very interesting / challenging test, but as I mentioned earlier, the instruments created are based on randomly changing boundaries – so the boundaries in the above example can change from 50%-50% to 30%-70% up/down movements(while creation of a specific “random instrument”).
there are several boundaries you can define that can effect the outcome of the “random data” instrument being created and tested.
After several millions of such tests you eventually get a very large and diverse range of testing data.
If you want to discuss the “random data” programs features farther feel free to contact me privately and I’ll be happy to do so over the phone / skype.
I estimate that if my System / theory have succeeded showing good results after testing on both random data in addition to real historical data of several hundreds of financial instruments – it is more likely to succeed in production

www.tickdata.com // best quality, high price
www.tickdatamarket.com // unknown, high price
www.cqgdatafactory.com // useful for rare request
www.anfutures.com // only S&P E-Mini,Nasdaq Index E-Mini, eFX CME
iqfeed.net // best price to quality 4 months of tick data, only USA
www.kibot.com // USA
www.grainmarketresearch.com // futures, nasdaq stocks
professional.teletrader.com // need request

Generally the system developers I come across use random data as way to invalidate, not validate their models, although your method seems to incorporate some trend persistence. This should make it tradeable but I am still concerned about so much information being missing.

For example, markets I find are innately perverse, in that the more money they give you in the shorter period of time the harder they try to take it back from you. This is borne out by runs tests on models and why the Turtles where given the “Last Trade a Loser” rule. As well as standing aside from markets a longer period of time, the larger the profit they had recently made. How do you incorporate these nuances into your artificial data?

hat’s helpful. Just one more bother because I’ve never doe live
trading high frequency ( if you want to call 15-30 minutes high frequency ) wise:

If you sign up with one of these trading vendors like tradestation or interactive brokers,
can you go back say a full year and get all the data for say top 1200 US cap ? It’s
down the road anyway ( once I’m done with what I’m building. right now I use stale data
from 2008 ) but I’d like to not have to pay say 10-15K for one year. Thanks a lot.

Appreciating that you haven’t gone live, there are issues your question raises you haven’t encountered yet.

From the top, with TradeStation.com and Interactive Brokers you can get a year’s history. If you want more history through IB you will want to get an eSignal feed which should allow you about 3 years.

Next issue… these ‘retail’ feeds are often limited to about 50 quotes per second, so you need to balance how short term you are trading versus the number of issues you’re following. You can speed this up with IB to about 150 quotes per second by going with their Computer To Computer Interface (CTCI), but you will need your own proprietary solution to interface through FIX.

Also, if you are using packages like TradeStation or MultiCharts to place trades electronically you have to realise that “EasyLanguage / PowerLanguage” do not support “Stop-Limit” order types to manage slippage (whereas Ninja-Trader does for example).

This is can be a MAJOR problem trading short term where your slippage can easily overwhelm you average trade. So, it is tough using these package to run live intraday given these limitation.

Given that, what happens if you consider increasing your time-frame to increase your average trade and ability to keep up with quotes and you expand the number of issues to follow to improve diversification…

Holding overnight is really fraught with a whole other set of issues as these packages do not maintain a full dialogue regarding your open positions. For example, if you get a glitch you can have all you automated models turned off (or TS just locks and you need to shut down and restart)…

Let’s say you have 20 open positions. When you reinstate the system automation the market-systems will be totally unaware of the existing open positions and will start placing trades as if you have a clean slate. So, you are left having to managing all your remaining open positions manually – which raises the question as to how you are supposed to do that when all your signals are internal to your automated system.

So, next you may try the move to your own custom written interface to the broker. Once again, you will find problems with availability of simulation platforms to test your code and flaky API’s, even with some of the higher-end applications like X-Trader. Even with vendors like Bloomberg we’ve had issues with poor change management. So don’t just accept the vendor upgrades – you should always test them locally.

Bottom line… most of the solutions are not all they are cracked up to be and it can take a while to find which strengths you need and which short-comings you can live with.

I as well am using random data tests in order to invalidate, not validate my model.
If I find that the model is not stable with any of the tests (historical or random data) – it’s back to the drawing board !

You wrote : “markets I find are innately perverse, in that the more money they give you in the shorter period of time the harder they try to take it back from you”.
I have to say that your approach to financial markets as an entity is very interesting, however I don’t see it that way…

I have to admit I am not familiar with the “Last Trade a Loser” rule you mentioned, does it refer to High Frequency traders that are competing for speed ? where only the faster one will “win” the deal ?

In any case – speed in that sense is not relevant for my model, my models in general are not aimed to compete for “millisecond deals”.
Sure, my execution system must be fast and precise in order to execute the dozens of trades it needs to execute per minute (at peak points), but if it will take 500 ms or even 1 second of delay occasionally – no harm will be done.
The model is based on a lot of small deals that are not dependent on one another, so even if the system will miss a deal completely – no big damage is done. it will never “chase” a deal, if a limit order the system placed is not met – it will simply delete it and move forward.
I do however have a potential slippage problem regarding closing of trades, to deal with this issue I am taking in to account a certain percentage of slippage when testing my models theoretically.

If you want the most reliable data for Interbank FX testing you would have to go to factset. They provide data that is that granular. I do believe that if you want to test at that level, you must have bid ask data. I have collected and scrubbed bid ask data on the majors for at least a two year period. This is more accurate than tic which can only provide a last price or bid or offer. In any of these cases it is not accurate enough to test a short term algo

You say you tested your EA on MT4 and you wish to keep on testing it in order to find eventual flaws. Then one thing you could do is to download historical data from all the MT4 brokers (Alpari, FXDD, FXCM, FXOpen, and so on) and test the EA with all those brokers. Why is that? Because every single broker gives you different historical and real-time data. And that’s not all, if you leave your MT4 platform opened to record data for a while, you will see it’s different form the data you can download from the same broker later on.
About the slippage problem, you should modify the closing condition in your code, or try a different broker. Only some brokers have that problem.