Quant development: Latency, Jitter, Throughput
How does your CORBA, JMS, RESTful, or “homegrown” messaging solution stack up? Note: I don’t work for RTI
Benchmarks show RTI Data Distribution Service leads the industry in performance and capacity.
Thanks for pointing out the capabilities of the OMG’s Data Distribution Service (DDS) for real time systems. It is a big improvement on traditional message queuing systems.
I don’t work for RTI either. I work for OCI we build and support open source implementations of well defined public specs for performance oriented systems.
We build OpenDDS which can be found here: www.opendds.org.
We also have extensive performance numbers which for the most part overlay those of RTI.Most of our tests have indicated we are running close to the theoretical maximum of the transports.Financial messages, even when aggregated are still at the low end of the message size for DDS less than 5 or 10k.
Over the years our testing with SGI on their systems has shown that hardware investments such as processor speed and transports are well leveraged by DDS technology.
For the person building out a financial system that is data/message intensive an open source implementation is shrewd move as they get insight into the overall performance and they can spend their money selectively on the hardware elements as they uncover the progressive bottlenecks as they grow their systems.
BTW we have been connecting OpenDDS to QuickFAST our open source FAST message decoder. Our latest benchmark on QuickFAST showed it decoding Opra at around 300 nanoseconds.
QuickFAST is a great front end to an event processing framework such as one we are implementing using OpenDDS. When you are doing intense analysis being able to spread the load with a flexible messaging system such as DDS provides you can get high throughput and lowest overall delay between feed, read and trade, in a cost effective manner.
What you save on licenses you can spend on hardware where it matters most.
regards Malcolm Spence
Director of Business Development
OCI St. Louis MO
I like that your DDS implementation is open source, but I don’t like that it uses the ace/tao broker for topics-subs-pubs database management. It’s a single point of failure
that is a common misunderstanding.
Our competition likes to imply with clients, that because we have an architecture that has an information repository, all messages must pass through that “info repo” and that therefore it poses a bottleneck and possible single point of failure.
This is not true.
1) the information repository can be federated so that subscribers etc. can in fact find publishers of topics from more than one source, in case anyone is unreachable for any reason. Federation also offers info-repo load balancing, and the ability to use colocated info-repos first. In secure messaging systems this ability to limit the number of info/repos is very important. They want that node to be a “trusted host” that also manages the keys for encryption. In many large scale communication systems there are only a few very secure , “trusted systems”.
2) Importantly in OpenDDS once a connection is made between a publisher and subscriber the data traffic flows DIRECTLY from one point to the other. (This is a key. First there is no longer any involvement by the InfoRepo. Secondly, in traditional message queuing products published messages go onto a queue, to which then a subscriber must then look to see if there is a message for them. This is additional latency.)
3) In OpenDDS, because there are only a few nodes that have the info/repo, it means that bandwidth consumed by making publisher and subscriber matches is small. The RTI model for example, is to use “many to many” multicast to make matches. This generates a lot more traffic and consumes bandwidth on messages that do not contain valuable content. Each time a new node joins many connections and messages are required.
4) OpenDDS has a pluggable transport framework. We can support various transports. This is important as many scalability issues are often more a transport problem than a DDS architecture problem. Using the correct transport, which closely matches the requirement characteristic, is an important first step.We are continuing to expand the transport framework for the Navy and this will expand options including other discovery models for users.
5) The info-repo happens to be a distributed application that was built using TAO. Actually the use of CORBA is hidden from the users. Our competition likes to imply you need to understand CORBA as well as DDS in order to use OpenDDS. Not so.
6) We just added Eclipse based open source modeling tools and code generators dramatically reducing the time to create an OpenDDS framework. We just used it on a financial project to rapidly build out an event processing framework.
I hope this explanation helps.
Regards Malcolm Spence
OCI St. Louis MO
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!