Quant Analytics New Discussion: I would like to compare results of algorithms. Is anyone interested in this topic?
Could be very good, in algorithmic trading there are often seemingly small implementation differences that make a large difference to the result. Comparing implementations is a good way to find out what’s important and what isn’t.
I’d say that the first problem would be to define a data set that you and others are interested in which is available to everyone. What’s the domain, indices, stocks, futures, forex? What’s the time scale, tick, second, minute, hour, day? Do you need just the “close” price or open, high, low and volume as well? If stocks, are they dividend adjusted?
You also need an agreed model of trading costs and slippage.
Once you’ve decided these, pick an algorithm you like and post the location of the data set and results. I’m sure you’ll find people working on the same algorithm and they’ll have different results so at least one party benefits in every difference resolution.
I am intrigued by your question.. because I was thinking of doing the same thing. However, I came across a recent article in a risk magazine where it seems that a researcher called Philip Maymin has approached this same problem. Just quoting the article verbatim:
In his paper, Philip Maymin argues that devising a trading algorithm is a ‘P’-type problem, while checking that it is profitable is ‘NP’. If the P = NP hypothesis is correct,
then checking strategies is fast enough for investors to be able to make the market
efficient by trading. If P does not equal NP, then the ability of investors to check all
strategies will be quickly overwhelmed by the exponential number of strategies there
are to check. Since a majority of computer scientists believe that P does not equal NP,
this is a sign that markets are not efficient and the stampede of the herd can sometimes
My question from the above is – does it make sense to compare results of algorithms? If any particular parameter-set or strategy wins out on one data set, it is the outcome of the data snooping problem. It is not necessary the strategy will win in another price set. assuming that your target is find the most profitable strategy..
My particular view would be to look at the minimal loss metric of each strategy under different simulations..
I commend you both for in-depth knowledge you have of the issues involved. I guess, for me, maybe ignorance is bliss. I use Ameritrade’s Strategy Desk only. They provide a language and all the connections to data. I have chosen to tap 5 minute data over the last 14 months (since July 10, 2011 to be exact) and keep writing different algorithms and discarding the ones that show lower profitable ones. I also keep an eye on number of transactions. I try to avoid day-trading by not selling what I bought on the same day. But if a position is sold from a prior day, then one round trip day trade can occur.
My favorite stock is LVS but the strategy applies to a wide swath of stocks. I would like to compare the results of trading LVS since July 10, 2011. Both LONG and SHORT plays. I can post my results here if you like.
Ah, well, you’ll have to count me out unless you can find a cheaply available version of the data that agrees exactly with what you’ve got (or close enough for you to switch).
, Ameritrade will give you all this for free. All you need is open a unfunded account and download StrategyDesk. Of course, the logic conversion may or may not be easy. For some functions I use 5 min interval data but for other functions I use a mix of intervals. Since 5 minutes if the lowest I use, that is what I declare at run time. Even 1 min is available.
Ameritrade seems to be US only at present. Even if I could open an account in general these platforms don’t allow you to export a their databases (for understandable reasons). That’s why I started by suggesting a domain which was popular and available.
I know you are probably not interested, but there are several sources of free intraday forex data so for the purposes of getting a community of people together to compare algorithms, that’s a possibility. Or there is very reasonably priced intraday indices and futures. If you would be prepared to compare your algorithms on some other data then hopefully the improvements you get from working with others will apply to your Ameritrade platform and stocks as well.
Maybe folks reading various books know the answer. What is considered to be
a good return on algorithmic trading and how frequent are the trades.. But
it would be better if people just take any stock and data of their choice
and indicate what the gross returns are. To start we can just exchange
information related to back tests. I dont mind sharing my information
(investment/return) if anyone is interested.
Isn’t the elephant in the room here: “Compare how?”What will be your basis for comparison, what will be your pain vs gain measure? There are as many measures of “goodness” as there are measurers! I happen to like CAGR / E(drawdown state) subject to minimum trade rate, minimum r^2 for the equity curve and minimum CAGR …. what do YOU like?
Perhaps you prefer some pareto measure with multiple objectives … but it is a great topic.
your comments almost went over the top here. LOL.
You are right, drawdowns are important. It was one of the factors I looked at in choosing Las Vegas Sands as the stock For a year it has been cycling between 36-53. So it controls the drawdowns to about 20%..
Where do we go from here?
a number of my candidates would be interested in speaking to you. We recruit traders and PMs into hedge funds and Investment Banks.. shall I put you in touch with them?
FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!