How to build Beowulf cluster aks HPC for Open MPI C++ apps like HFT with R with RCPP RInside package on Debian Linux #rstats #linux

(Last Updated On: November 16, 2012)

How to build Beowulf cluster aks HPC for Open MPI C++ apps like HFT with R with RCPP RInside  package on Debian Linux #rstats #linux

Install Debian Etch from: and follow first CD install instructions:

http://www.debian.org/CD/http-ftp/

Install Debian Virtual Machines in Virtual Box as usual

http://www.wired.com/geekdad/2012/02/debian-linux-on-oracle-virtualbox/

Go with all default options.  You can always clone  a second  client VM to create a slave.

Set up hostname and configure network via: (choose the static address  not DHCP)

http://www.debian-administration.org/articles/254

Note that you could loose network connectivity soo add an additional Network Adapter with Bridged Adapter for each VM. within the Virtual Box setting. so see next step

Ensure each VM has network adapter setting to Virtual Box Host Only

http://christophermaier.name/blog/2010/09/01/host-only-networking-with-virtualbox

Create NFS for both virtual machines as server and client:

http://www.howtoforge.com/nfs-server-and-client-debian-etch

Note: When  editing exports on the server with:

/home           192.168.0.101(rw,sync,no_root_squash)

/var/nfs        192.168.0.101(rw,sync)

 

Note that 101 is the client!. ALso, I have when I try to auto mount during boot on the client, it would hang. I just created a shell script to do the mounts manually when you boot.  I just called this /etc/mountmaster.sh

 

If you need to add users do:

http://www.debian-administration.org/articles/2

You should be able to proceed with the following set up of your Beowulf cluster:

http://www.debian-administration.org/articles/2

To understand nad configure ssh connections with no passowrd, use:

SSH your Debian servers without password

This could be useful http://wiki.debian.org/SSH but note it uses RSA not DSA

If confused, use this next one but remember the ssh server is the Open MPI slave while client is the Open MPI since it logins into the Open MPI slave.

http://www.howtoforge.com/set-up-ssh-with-public-key-authentication-debian-etch

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!
This entry was posted in Featured, HFT High Frequency Trading, Premium Membership, Quant Development, R and tagged , , , , , , , , , , , , , , on by .

About caustic

Hi i there My name is Bryan Downing. I am part of a company called QuantLabs.Net This is specifically a company with a high profile blog about technology, trading, financial, investment, quant, etc. It posts things on how to do job interviews with large companies like Morgan Stanley, Bloomberg, Citibank, and IBM. It also posts different unique tips and tricks on Java, C++, or C programming. It posts about different techniques in learning about Matlab and building models or strategies. There is a lot here if you are into venturing into the financial world like quant or technical analysis. It also discusses the future generation of trading and programming Specialties: C++, Java, C#, Matlab, quant, models, strategies, technical analysis, linux, windows P.S. I have been known to be the worst typist. Do not be offended by it as I like to bang stuff out and put priorty of what I do over typing. Maybe one day I can get a full time copy editor to help out. Do note I prefer videos as they are much easier to produce so check out my many video at youtube.com/quantlabs