fbpx

Craigslist.org uses Redis Sharding and PERL to get fast NOSQL cluster database with 1 ms low latency. Note for HFT platforms!

(Last Updated On: June 4, 2012)

Craigslist.org uses Redis Sharding and PERL to get fast NOSQL cluster database with 1 ms low latency. Note for HFT platforms!

Here are some highlights why I think Redis is the way to go for any ultra low latency HFT platform:

we prefer to have predictibly low latency. It is common to see calls complete in 1ms or less. Many of our calls have timeouts of 100ms to handle exceptional cases. Insetad of frequent disk writes, we replicate every node to a slave node on a different physical host and give it an otherwise identical configuration.

They also code in PERL with examples:

The bulk of our code is Perl, so the configuration is expressed in Perl for easy consumption. Here’s a minimal example that uses 2 hosts, assumed to have 2 cores each. This provides 4 total redis nodes, two primaries (masters) and two secondaries (slaves). Losing a single host will result in virtually zero data loss.

Unique hashing capabilities:

By mapping keys to node names instead of directly to nodes (host:port pairs), we have the freedom to relocate a node to a different server without disturbing the consistent hashing ring.

By mapping keys to node names instead of directly to nodes (host:port pairs), we have the freedom to relocate a node to a different server without disturbing the consistent hashing ring.

Interesting indeed for any one looking at doing a highly fast HFT platform. Keep this in mind regardless of what those trendy hipsters may say.

From: http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/

Another interesting article:

http://antirez.com/post/redis-presharding.html

I always have been told, if you want ultra fast, do sharding.

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Subscribe For Latest Updates

Sign up to best of business news, informed analysis and opinions on what matters to you.
Invalid email address
We promise not to spam you. You can unsubscribe at any time.

Join our Private Chat Server for FREE Trading signals for ETH DOGE ADA

Scroll to Top