Seeking quant development solutions for replay & back-testing on large sets of captured full depth market data

(Last Updated On: March 27, 2012)

Seeking quant development solutions for replay & back-testing on large sets of captured full depth market data

Seeking solutions for replay & back-testing on large sets of captured full depth market dataMy firm is looking to upgrade our proprietary trading strats/systems from using top-of-book only data to instead using full depth of book market data in the hopes it will help us make more informed trading and order sizing/routing decisions and are interested in hearing other people’s experiences making such a transition, and in particular what solutions are out there for replaying full depth data in our research and backtesting environment. We have been capturing the depth data from all “lit” N.American Equity Exchanges for over a year and now have over 5TB of it captured in binary format. Ideally we would like to be able to utilized our existing captured data and replay it on demand in an efficient way without having to put much time/resources towards developing our own system to do so. What sorts of solutions are others using to do this? Interested in both hardware and software aspects, with a priority towards the depth data replay/backtesting capabilites, however if the system(s) can also be used in low latency realtime production environment, and/or have reporting or CEP capabilities that can aid in R&D, that certainly doesn’t hurt.
==-Have you looked at OneTick? My company produces SSD based hardware to act as either the application engine for these types of applications, or we can also configure as iSCSI storage for another server and provide over 3GB/s bandwidth and up to 600,000 IOPS from a single 2u box at under 500Watts. 10TB of SSD storage per system. Let me know if you would like more data or check out our url @ —
For speed and CEP, You might have a look on these ones : -StreamBase -TimesTen -kdb+ -Sybase ASE -Gemstone, Vhayu (mentioned too but We never tried these two last) You might need a bit of TIME to assess ,choose and setup the whole thing

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!
This entry was posted in Quant Development and tagged , , , , , , , on by .

About caustic

Hi i there My name is Bryan Downing. I am part of a company called QuantLabs.Net This is specifically a company with a high profile blog about technology, trading, financial, investment, quant, etc. It posts things on how to do job interviews with large companies like Morgan Stanley, Bloomberg, Citibank, and IBM. It also posts different unique tips and tricks on Java, C++, or C programming. It posts about different techniques in learning about Matlab and building models or strategies. There is a lot here if you are into venturing into the financial world like quant or technical analysis. It also discusses the future generation of trading and programming Specialties: C++, Java, C#, Matlab, quant, models, strategies, technical analysis, linux, windows P.S. I have been known to be the worst typist. Do not be offended by it as I like to bang stuff out and put priorty of what I do over typing. Maybe one day I can get a full time copy editor to help out. Do note I prefer videos as they are much easier to produce so check out my many video at youtube.com/quantlabs