Tag Archives: Erlang

More HFT talk on Erlang RDMA networking for ultra lowest latency

More HFT talk on Erlang RDMA networking for ultra lowest latency

Update talking info since the last few days for further HFT analysis

Some history:

https://quantlabs.net/blog/2014/12/c-secret-vendor-and-open-source-libraries-potentially-used-for-hft-environments-thanks-to-this-barclays-bank-presentation-video/

Erlang from Barclays Bank Go Big video with weaknesses including potential RDMA open source solution
Pros and cons are covered in the video below. The questions is, does this:

Adventures in HPC: RDMA and Erlang


Provide a solution or workaround satisfactorily?

https://github.com/MrStaticVoid/rdma_dist

Also, Goldman Sachs is confirmed using Erlang in their HFT environment

http://en.wikipedia.org/wiki/Erlang_%28programming_language%29

Learn more if Erlang can be used in an HFT environment

The video

http://www.informatix-sol.com/cloud.html

It’s getting increasing difficult to utilize these with conventional programming languages such as Java, C# or C++. The threading and mutex’s soon become a nightmare to code and to debug. We are finding that the concurrency of Erlang is a good fit for Cloud and we now use this by choice.

http://www.informatix-sol.com/docs/ScaleErlang2.pdf

https://nativecoding.wordpress.com/high-performance-gamefinance-tech-resources/

Netmap is commonly used in HFT as well as packet filtering applications. I believe Verisign is running some of the root DNS servers with netmap as well, getting millions of connections per second.

HFT user userland based IP stack

https://news.ycombinator.com/item?id=9726185

Example libraries?

https://software.intel.com/en-us/blogs/2015/06/12/user-space-networking-fuels-nfv-performance

https://github.com/pkelsey/libuinet (from above Intel list)

http://www.eecs.berkeley.edu/~sangjin/2013/01/14/NUSE.html

Netmap setup

http://ytliu.info/blog/2014/05/28/netmap-setup

Intel DPDK

http://www.intel.com/content/dam/www/public/us/en/documents/presentation/dpdk-packet-processing-ia-overview-presentation.pdf

JOIN MY FREE NEWSLETTER TO LEARN HOW HFT TECHNIQUES CAN BENEFIT YOUR ALGO TRADING

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Go or Elgang will go anywhere for HFT

Go or Erlang will go anywhere for HFT

These hot cloud and web languages always need to compare themselves to performance to C so I will stick with the older legacy languages like c/c++ Java languages. I do think Docker looks interesting but how do you use it for a trading environment?

http://www.cio.com/article/2985056/developer/rating-programming-languages-swift-is-hot-dart-is-not.html

Join my FREE newsletter to see which language I stick with over the long runĀ 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

The death knell to this Erlang with HFT in mind just knocked, oh hello C or C++ AGAIN How did we forget about you?

The death knell to this Erlang with HFT in mind just knocked, oh hello C or C++ AGAIN How did we forget about you?

The other technology to keep in mind is high speed ZeroMQ for message service bus.

Here were some untouched selling point:

http://www.quora.com/Where-and-how-is-Erlang-used-in-the-finance-industry

http://www.erlang.org/workshop/2008/Sess23.pdf (refer to page 8 with DLL library loading.

Note comments like this from you know know who:

I do like the concurrency features of Erlang but I think it will struggle to compete with the C++ based Algo engines

https://www.linkedin.com/groups/Erlang-Algorithmic-Trading-60972.S.230321790]

That is what we call the final death knell to this Erlang etc thought.

This fizzes out here to:

https://www.linkedin.com/groups/Erlang-Algorithmic-Trading-Open-Source-90878.S.231879090

Not interested in this but the Matlab or R plugin could be useful for someone

https://github.com/henry-hz/erlang-trader

In terms of FIX, this does not excite me:

http://www.ieiss.com/

Interesting chart for performance vs the type of trading strategy

http://www.quantpedia.com/Chart/Performance

Join my FREE newsletter to learn more the right away to develop these potential HFT systems

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Uknown HFT secret? Erlang uses NIFS with C to deliver trading strategy logic?

Uknown HFT secret? Erlang uses NIFS with C to deliver trading strategy logic?

This was pointed by an ‘insider’ as something to further investigate. I am wondering how you see the potential of this so comment away

http://www.erlang.org/doc/tutorial/c_portdriver.html

http://stackoverflow.com/questions/26773125/erlang-nifs-threads-locking

http://erlycoder.com/59/erlang-how-to-integrate-c-with-erlang-program-example-or-erlang-nifs-native-implemented-functions-us

Join my FREE newsletter to about the potential secrets of Erlang for HFT use

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Programming Language Lessons learning about Barclays HFT use cases including Java C++ Linux FPGA and Erlang

Programming Language Lessons learning about Barclays HFT use cases including Java C++ Linux FPGA and Erlang

Below is an eye popping but very recent useful video about HFT in Barclays. These topics were covered:

Java is not as desirable due to garbage collection which leads Linux Operating system Jitter

FPGA is complicated to debug and deploy

C++ techniques and libraries matter to accomplish your lowest latency

Direction towards NOSQL solutions of housing data including Cassandra and MongoDB (C++) highlighted

Erlang could be the most useful functional programming languages

Erlang from Barclays Bank Go Big video with weaknesses including potential RDMA open source solution

Note that Python nor R are mentioned in this conversation

Network monitoring matters for dropped data packets

Barclays HFT uses with vendor panel including Solarflare and Corvil video: Decent Open source solutions?

A couple of outstanding questions remain with what I am learning thus far

Join my FREE newsletter to learn the right ways how to deploy HFT

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Erlang from Barclays Bank Go Big video with weaknesses including potential RDMA open source solution

Erlang from Barclays Bank Go Big video with weaknesses including potential RDMA open source solution
Pros and cons are covered in the video below. The questions is, does this:

Adventures in HPC: RDMA and Erlang


Provide a solution or workaround satisfactorily?

https://github.com/MrStaticVoid/rdma_dist

Also, Goldman Sachs is confirmed using Erlang in their HFT environment

http://en.wikipedia.org/wiki/Erlang_%28programming_language%29

Learn more if Erlang can be used in an HFT environment

The video

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Good starting point for Redis NOSQL and Erlang combo? Github repo of goodies

Good starting point for Redis NOSQL and Erlang combo? Github repo of goodies

This is what we call different but sounds quite compelling to work from with this video

https://github.com/cbd

Join my FREE newsletter to newsletter to learn more how I proceed with this combo

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Redis could be better choice for tradng over MongoDB NOSQL if you using Erlang but they got databases to (call me extra confused)

Redis could be better choice for tradng over MongoDB NOSQL if you using Erlang but they got databases to (call me extra confused)

The weakness could be overcome as you run multiple instances on this for Redis
http://redis.io/topics/partitioning

The deal breaker for Redis over Mongo:

Maybe Redis is better due to MongoDB Erlang drivers are community driven

http://docs.mongodb.org/ecosystem/drivers/

Erlang database options but never tested:

Why big companies use Mnesia instead of using Riak or CouchDB – Stack Overflow

http://stackoverflow.com/questions/23180484/why-big-companies-use-mnesia-instead-of-using-riak-or-couchdb

3 database for Erlang Mnesia Riak and CouchDB
https://www.erlang-solutions.com/products/riak-nosql-database

(looking for opinions here)

historical trend of the popularity ranking of database management systems

http://db-engines.com/en/ranking_trend

Join my FREE newsletter to learn more about why way I go with a database

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Check out this video for a virtual 3D tour of all your processes running in an Erlang server?

Check out this video for a virtual 3D tour of all your processes running in an Erlang server?

This is pure awesomeness but I wish something like this also existed for an entire Linux server. Anyhow, this is one more reason why I am really digging this Erlang

Join my FREE newsletter to learn more about this powerful language

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Is Java Erlang Python stack better than WIndows DotNet with Csharp and Fsharp for automated trading

Is Java Erlang Python stack better than WIndows DotNet with Csharp and Fsharp for automated trading with quant analysis

This is tought decision moving forward for the back end side of this system. It try to explain in this video

Is Java 8 much faster than 7? It seems so with these lambda calls. Does Linux server help vs Windows ?

 

My Meetups on this:

http://www.meetup.com/quant-finance/events/218915370/

http://www.meetup.com/R-Matlab-Users/events/218915338/

http://www.meetup.com/Toronto-NET-Developer-Meetup/

Other links:

http://www.quora.com/How-can-I-call-functions-between-Python-Java-and-Erlang

http://thrift.apache.org/

http://www.eprosima.com/index.php/en/

http://stackoverflow.com/questions/69316/biggest-differences-of-thrift-vs-protocol-buffers

http://www.erlang.org/doc/apps/jinterface/java/com/ericsson/otp/erlang/OtpConnection.html

http://www.erlang.org/doc/apps/jinterface/jinterface_users_guide.html

https://quantlabs.net/blog/2014/12/is-java-8-much-faster-than-7-it-seems-so-with-these-lambda-calls-does-linux-server-help-vs-windows/

Not mentioned in video: Even HFT brokerages like Lime only support Java or C++ in Linux

Join my FREE newsletter to see which way I go

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!