Posts Tagged ‘Java’
Youtube video on How to populate a highly scalable NOSQL Redis database from Java with source code
Youtube video on How to populate a highly scalable NOSQL Redis database from Java with source code
Some links to you out:
Usage:
https://github.com/xetorthio/jedis
Download the JAR:
https://github.com/xetorthio/jedis/downloads
Learn more on how I do http://quantlabs.net/membership.htm
Best alternative development stack for R with Hadoop? Forget MYSQL? Cassandra and Java listeners for market tick data
View CommentsWritten on May 17th, 2012 by caustic
Categories: HFT High Frequency Trading, Quant Analytics, Quant Development, R
Categories: HFT High Frequency Trading, Quant Analytics, Quant Development, R
Best alternative development stack for R with Hadoop? Forget MYSQL? Cassandra and Java listeners for market tick data
After reading about the limitation of MYSQL and how expensive it can get, I decided to take chance on Cassandra. One big reason there is a RCassandra package within CRAN so yippee for that. Also, the install does not look hard and better yet, you can integrate it with Hadoop. Yipee for that! Also, Cassandra may be faster for writing than HBase which was part of the RHadoop offering so boo to that. Also, I plan to have to some Java listeners to my market data to populate the Cassandra database. This stack may work so let’s cross my fingers. Here are my links that got me thinking this way:
http://stackoverflow.com/questions/4884967/hadoop-hbase-hdfs-vs-mysql-or-postgres-loads-of-independent-structured-d
http://ria101.wordpress.com/2010/02/24/hbase-vs-cassandra-why-we-moved/
http://blog.milford.io/2010/06/installing-apache-cassandra-on-centos/
This above install of Cassandra appears to work with a few tricks as running as root but does work and installed fine.
http://www.datastax.com/docs/0.7/map_reduce/hadoop_mr
http://code.google.com/p/cassandra-java-client/
The last 2 links I question but the Cassandra install may be worth doing but integrating with Hadoop could be a challenge. I also hope the RCassandra works to.
So many thanks goes to powerful open source free software for CentOS Enterprise Linux, Java, R, RStudio, and Hadoop with Cloudera
View CommentsWritten on May 16th, 2012 by caustic
Categories: HFT High Frequency Trading, Quant Development, R
Categories: HFT High Frequency Trading, Quant Development, R
So many thanks goes to powerful open source free software for CentOS Enterprise Linux, Java, R, RStudio, and Hadoop with Cloudera
Learn more her eon what I do
http://quantlabs.net/membership.htm
Open source stuff with Java, Hadoop, Linux, R. Future Meetups with algo and strategies?
View CommentsWritten on May 15th, 2012 by caustic
Categories: Premium Membership, Quant Development, R
Categories: Premium Membership, Quant Development, R
Hi there
I am doing a technical about face from Matlab and .NET. This could be temporary or permanent but depends on the results of my findings in open source. I don’t know but I am setting up a new infrastructure with CentOS Linux, Hadoop with Java, and R, and various R packages. Everything is open source and does not cost a penny. This was all driven to the heavy licensing fees Mathworks charges for clustering. As a result, I moved to this new option. Anyhow, I may do a future general Meetup on this.
Also, I have finished my own development of an HFT algo/strategy development course which is part of my QuantLabs.net Premium membership. Many people have asked questions about this but may focus this Meetup group on this sort of stuff. But that is down the line.
So, over the short term, I am wanting to network with some people who may be interested in sharing some strats and algos. I am constantly being approached by many capital advisors looking to throw around large amounts of cash due to the lack of low yield options out there. So, if you want let’s do it over the next two Tuesdays. I will tentatively create two physical Meetups from our usual spot in North York/Toronto. I will pick which appears to be the most popular week. I can also one on ones as well but let me know your story. I will need to know ASAP on either for reservation.
As for presentations, a few people would like to present some ideas over the summer. If you are one of them, please reach out to me as well. I am also looking at some online ones including a very promising open source Matlab automated trading system. It looks pretty sweet.
I may get a GotoMeeting.com account to do some online presentation as well both for QuantLabs.net Premium members and general visitors to various events.
Let me know what you think by responding to this via email or posting on Meetup. I Just want to let you know I have not forgotten about you all. I have been just been busy with this new infrastructure stuff.
Thanks Bryan
P.S. More info at Meetup.com/quant-finance or Youtube.com/quantlabs
Surprise! I thought Matlab Coder toolbox C++ Hadoop Pipes would be a good option vs Java, Hadoop, and R
Surprise! I thought Matlab Coder toolbox C++ Hadoop Pipes would be a good option vs Java, Hadoop, and R
Another option is to drop R, replace with C++ generated code from Matlab using the Coder toolbox. You can then pipe through using Hadoop Pipes. This is just an idea but could work but everything depends on the kinds of C++ code generated by Matlab. It does eliminate the need of needing to learn a language like R which could be a huge battle with some undocumented packages. This seems to be the big gripe of R.
Here are some tutorial links with C++ and Hadoop Pipes.
http://developer.yahoo.com/hadoop/tutorial/module4.html
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/pipes/package-summary.html
But this lead me to this link:
http://cs.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C%2B%2B_Programs_on_Hadoop
The wordcount program in native Java, in Python streaming mode and in C++ pipes mode is run on 6 books from the Gutenberg project:
Damn! C++ Hadoop Pipes seems twice as slow as Java. I can never win.
Method Real Time
(seconds) Ratio to Java
Java 2 min 15.7 1.0
C++ 5 min 26 0.416
Python 12 min 46.5 0.177
Considering free open source R, Java, Hadoop, with direct LMAX connection for HFT with helpful links
View CommentsWritten on May 8th, 2012 by caustic
Categories: HFT High Frequency Trading, Quant Development
Categories: HFT High Frequency Trading, Quant Development
Considering free open source R, Java, Hadoop, with direct LMAX connection for HFT with helpful links
I have been exploring various options versus my open source HFT platform with Matlab. It seems there might be some complication from a clustering point of view with expensive licensing on Matlab and Windows products. As a result, I have looked around for new options and Hadoops always popped up. Also, Matlab does not integrate too well with Hadoop so my next option was for R for analytics and prototyping. I have lots to learn with R if I go this option. I know Java so I am ok there. Java RTS could be a good option here but can always involve a C++ platform if need be but this is still undecided.
Another challenge was the broker but after seeing LMAX in UK , I was convinced that this might be the broker since they are market neutral. This might be the way to go in terms of a a high frequency trading stack. Remember this is all true open source as well which mean it is free!
Here are some links to help get you started:
Linux but will use CentOS for the distribution with Hadoop:
http://vimeo.com/8615628 <-- he uses Arch but makes valid points on it
Hadoop:
http://stackoverflow.com/questions/7053837/video-tutorial-for-hadoop
There are quite a few Youtube demos to show the power and quickness of this.
R with Hadoop (RHadoop):
http://www.r-bloggers.com/big-data-analytics-with-r-and-hadoop/
http://www.youtube.com/watch?feature=player_embedded&v=QEaOfTuveGg
ActiveQuant could be a good option for an API as well.
ActiveQuant could be a good option for an API as well.
Linux or Unix Shell commands to get Active Quant AQ2o going for potential Java High Frequency Trading environment
Linux or Unix Shell commands to get Active Quant AQ2o going for potential Java High Frequency Trading environment
These are commands sent from Ulrich (founder of Active Quant) of Active Quant to get the current AQ2o server going. Let’s hope this works. If so, I will include a Youtube video within my Linux environment. Thanks go out to him.
Update: I am not a Linux admin expert, but it can be frustrating and wonky to work with. As in the past, I need to stick with Windows based systems for productivity reasons. Linux painfully slows me down but these commands seem to work but the environmental handling of Linux is a nitemare so good luck to anyone implementing these commands. My opinion of Linux has no reflection on Active Quant itself.
------
More details at http://activequant.org/Below you'll find some unix commands that will get you going. ==============
# AQ2o is good in a large corporation. Because it is using hbase and hadoopunderneath, it can scale to petabytes of storage. # Use the server as a centralstorage point. The OpenCore that you use is not good for backtesting ... # mkdir presswork cd presswork # download the aq2server wget "http://downloads.sourceforge.net/project/activequant/aq2o-2.1-SNAPSHOT-bundle.zip?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Factivequant%2Ffiles%2F&ts=1334934472&use_mirror=heanet" -O aq2o.zip # unzip aq2o.zip cd aq2o sh runserver.sh # press CTRL-C to stop the server # modify aq2server.properties to enable thesoap server # soapserver.start=true sh runserver.sh # when things are settled ... open a web browser and have a look at --->http://localhost:9999/main?wsdl That's the WSDL for the soap service, have alook http://users.skynet.be/pascalbotte/rcx-ws-doc/dotnet.htm # leave the server running ... let's pump in data. cd .. svn co http://activequant.org/svn/aq2o/trunk aq2o_trunk mvn package -DskipTests=true export AQ_HOME=/home/ustaudinger/presswork/aq2o_trunk cd trunk/src/test/resources/sampledata sh /home/ustaudinger/presswork/aq2o/importInstruments.sh sh/home/ustaudinger/presswork/aq2o/importMDIs.sh #
# connect with Squirrel SQL and have a look at things ...
# now let's have a look at the data in our archive ... R x = read.csv("http://localhost:44444/?SERIESID=YAHOO_cbk.de&FREQ=EOD&FIELD=CLOSE&STARTDATE=20010101&ENDDATE=20120301")plot(x[,2]) B
Youtube video demo of Quant membership with courses on algo, strategy development, HFT, QuantLib, C++, Matlab, C#, Java
View CommentsWritten on April 21st, 2012 by caustic
Categories: HFT High Frequency Trading, Matlab, Premium Membership, Quant Development
Categories: HFT High Frequency Trading, Matlab, Premium Membership, Quant Development
Youtube video demo of Quant membership with courses on algo, strategy development, HFT, QuantLib, C++, Matlab, C#, Java
http://quantlabs.net/membership.htm
Quant membership benefitis here:
http://quantlabs.net/quant-member-benefits/
Youtube video demo of Quant membership with courses on algo, strategy development, HFT, QuantLib, C++, Matlab, C#, Java
View CommentsWritten on April 21st, 2012 by caustic
Categories: Matlab, Premium Membership, Quant Development
Categories: Matlab, Premium Membership, Quant Development
Youtube video demo of Quant membership with courses on algo, strategy development, HFT, QuantLib, C++, Matlab, C#, Java
http://quantlabs.net/membership.htm
Quant membership benefitis here:
http://quantlabs.net/quant-member-benefits/
Open source trading platforms including Active Quant with Ubuntu Linux Java demo
Open source trading platforms including Active Quant with Ubuntu Linux Java demo
http://quantlabs.net/membership.htm This existed once upon a time. Point of this video was Linux is frustrating to use. I am sticking with Windows as a result to save time and money on painkillers.
