Tag Archives: Cluster

For HFT: Nvidia CUDA Visual C++ financial examples show power of GPU versus horrendous cluster hardware management

For HFT: Nvidia CUDA Visual C++ financial examples show power of GPU versus horrendous cluster hardware management

With the research below it seems that CUDA may be the easiest way to go working off the supplied CUDA examples for financial needs. There will be no need of clustering, servers, etc which will make life so much easier. The outstanding item is to see C++ CUDA examples with trading signals which I am now seeking.

1. Download CUDA toolkit https://developer.nvidia.com/cuda-downloads. I am using Windows 7

2. Import the samples into Visual Studio 2010 and look at the C++ Financial samples

Note on how CUDA is much faster than Open GL espeecially on GeForceGTX 480. Below is used for technical indicators.

http://www.amazon.com/GeForce-1536MB-Express-Graphics-ZT-40101-10L/dp/B003DTKU82

http://www.elucidare.co.uk/assignments/project_gpufinance/abst_gpufinance.php

http://www.elucidare.co.uk/assignments/project_gpufinance/16thieeeacmdsrt_submission_34.pdf

The listed GeForce board is quite affordable as well for $340 for 480 cores. Nice!

Note: I have dropped the idea of Fastflow with Visual C++> The author did not bother showing how to use Cmake to build the library so I dropped the idea. I really don’t care about the benchmark anyhow as I just want to see it work as fast as possible.

 See how I proceed with this to build my HFT platform by joining my newsletter

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to build Beowulf cluster aks HPC for Open MPI C++ apps like HFT with R with RCPP RInside package on Debian Linux #rstats #linux

How to build Beowulf cluster aks HPC for Open MPI C++ apps like HFT with R with RCPP RInside  package on Debian Linux #rstats #linux

Install Debian Etch from: and follow first CD install instructions:

http://www.debian.org/CD/http-ftp/

Install Debian Virtual Machines in Virtual Box as usual

http://www.wired.com/geekdad/2012/02/debian-linux-on-oracle-virtualbox/

Go with all default options.  You can always clone  a second  client VM to create a slave.

Set up hostname and configure network via: (choose the static address  not DHCP)

http://www.debian-administration.org/articles/254

Note that you could loose network connectivity soo add an additional Network Adapter with Bridged Adapter for each VM. within the Virtual Box setting. so see next step

Ensure each VM has network adapter setting to Virtual Box Host Only

http://christophermaier.name/blog/2010/09/01/host-only-networking-with-virtualbox

Create NFS for both virtual machines as server and client:

http://www.howtoforge.com/nfs-server-and-client-debian-etch

Note: When  editing exports on the server with:

/home           192.168.0.101(rw,sync,no_root_squash)

/var/nfs        192.168.0.101(rw,sync)

 

Note that 101 is the client!. ALso, I have when I try to auto mount during boot on the client, it would hang. I just created a shell script to do the mounts manually when you boot.  I just called this /etc/mountmaster.sh

 

If you need to add users do:

http://www.debian-administration.org/articles/2

You should be able to proceed with the following set up of your Beowulf cluster:

http://www.debian-administration.org/articles/2

To understand nad configure ssh connections with no passowrd, use:

SSH your Debian servers without password

This could be useful http://wiki.debian.org/SSH but note it uses RSA not DSA

If confused, use this next one but remember the ssh server is the Open MPI slave while client is the Open MPI since it logins into the Open MPI slave.

http://www.howtoforge.com/set-up-ssh-with-public-key-authentication-debian-etch

 

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to set up a C++ Beowulf cluster for Open MPI for Rinside with VirtualBox and Ubunutu Linux #linux

How to set up a C++ Beowulf cluster for Open MPI for Rinside with VirtualBox and Ubunutu Linux

It looks like to implement the RInside C++ call into the FastFlow will not be fun. As a result, I am revisiting OpenMPI as RInside’s sample code already includes a sample with it implemented. I now focus how build how to build a virtual home network Beowulf cluster with VirtualBox:

1. Set each virtual machine with Ubuntu Server slave nodes and a Ubuntu Desktop for your master node.

2. Set each with a static IP in Virtual Box.

http://www.coding4streetcred.com/blog/post/VirtualBox-Configuring-Static-IPs-for-VMs.aspx

3. Do the rest here:

http://byobu.info/wiki/Building_a_simple_Beowulf_Like_Cluster_with_Ubuntu

Find out more what I do

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Youtube video: NOSQL Redis data base cluster to parallelize demo in R for trading model or profitable strategy forecasting

Youtube video: NOSQL Redis data base cluster to parallelize demo in R  for trading model or profitable strategy forecasting

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Live demo event: Parallelize your R analysis big data and use FREE open source NOSQL as a potential market database cluster

Live demo event: Parallelize your R analysis big data and use FREE open source NOSQL as a  potential market database cluster

People have asked for things including how to create a ticker factory, store market data, and how to quickly parallelize the data for analytical processing.  I will show a complete open source NOSQL database that can be used for panellizing processes and a data repository. This includes failover, clustering, and redundancy as well. This is based on a master and multi slave relationship which will be demoed. This is a very easily implemented solution to use as well as little moving parts to enable simple maintenance compared to other  popular big data NOSQL solutions. The client will be demoed within a R script.  This NOSQL solution is so impressive even Microsoft has an implementation of it in its new Open Tech community.

Note: You will need access into the QLN Premium membership to get access to this webinar!

https://quantlabs.net/blog/complete-calendar/

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant analytics: Live webinars coming soon on capturing market data, NOSQL to cluster, best fit model with ARIMA. R script included

Quant analytics: Live webinars coming soon on capturing market data, NOSQL to cluster, best fit model with ARIMA.  R script included

 

he live demo and webinars for my members are getting works out:

1. How to capture market data fast with an affordable service

2. Parallelize your analysis and use open source NOSQL for potential tick data repository

3. How to do best fit modelling and implement ARIMA processes and simulations. My next models will include GARCH.

Get instant access to understand the details and be part of theses.

Also, for anyone who has general Q&A about what I do.

Check here for the upcoming schedule

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Youtube video demo on NOSQL Redis on Windows and Linux clusterto parallelize with R and doRedis package

Youtube video demo on NOSQL Redis on Windows and Linux clusterto parallelize with R and doRedis package

http://quantlabs.net/r-blog/2012/07/will-microsoft-opentech-put-redis-into-1-status-for-in-memory-nosql-database-for-linux-window-and-mac-os-x/

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Youtube video: Moving Average R Code with Open Source HFT C Linux trading platform coming soon with MYSQL Cluster

Youtube video: Moving Average R Code with Open Source HFT C Linux trading platform coming soon with MYSQL Cluster

http://quantlabs.net/membership.htm

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Craigslist.org uses Redis Sharding and PERL to get fast NOSQL cluster database with 1 ms low latency. Note for HFT platforms!

Craigslist.org uses Redis Sharding and PERL to get fast NOSQL cluster database with 1 ms low latency. Note for HFT platforms!

Here are some highlights why I think Redis is the way to go for any ultra low latency HFT platform:

we prefer to have predictibly low latency. It is common to see calls complete in 1ms or less. Many of our calls have timeouts of 100ms to handle exceptional cases. Insetad of frequent disk writes, we replicate every node to a slave node on a different physical host and give it an otherwise identical configuration.

They also code in PERL with examples:

The bulk of our code is Perl, so the configuration is expressed in Perl for easy consumption. Here’s a minimal example that uses 2 hosts, assumed to have 2 cores each. This provides 4 total redis nodes, two primaries (masters) and two secondaries (slaves). Losing a single host will result in virtually zero data loss.

Unique hashing capabilities:

By mapping keys to node names instead of directly to nodes (host:port pairs), we have the freedom to relocate a node to a different server without disturbing the consistent hashing ring.

By mapping keys to node names instead of directly to nodes (host:port pairs), we have the freedom to relocate a node to a different server without disturbing the consistent hashing ring.

Interesting indeed for any one looking at doing a highly fast HFT platform. Keep this in mind regardless of what those trendy hipsters may say.

From: http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/

Another interesting article:

http://antirez.com/post/redis-presharding.html

I always have been told, if you want ultra fast, do sharding.

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Youtube algo strategy course with programming and new HFT Matlab cluster set up

Youtube algo strategy course with programming and new HFT Matlab cluster set up

http://quantlabs.net/quant-member-benefits/slash-your-quant-learning-curve/

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!