Tag Archives: Message Queuing

Oracle 12c successfully install for my requirements for high speeding trading with message queuing in memory and more features

Oracle 12c successfully install for my requirements for high speeding trading with message queuing in memory and more features

Sepcial NOTE: on installation: The Server install seems to be work ok but the Workstation some wierd permissioning problems for the database when installed.

Successfully and easily installed on my Windows 2012 Server. It is a strange world we live in now that it sort of works. After digging through, Oracle 12c could meet all my requirements.

Why Oracle?

When you read about Liquidnet being used, it got me wanting to try it out.

My REAL reasons why Matlab is important to me thanks to Mr FPGA real time charting and ease of use for FAST quant trading idea generation

 

http://wedostreams.blogspot.ca/2009/01/oracle-aq-buffered-queues-101.html?m=1

Well, boy was I surprised but here is what I found:

Performance:

Oracle local is faster than TimesTen

http://zenodo.org/record/7566/files/CERN_openlab_report_Endre_Andras_Simon.pdf

http://en.m.wikipedia.org/wiki/TimesTen

In Memory TimesTime database functionality

http://www.peakindicators.com/index.php/knowledge-base/1136-performance-benchmarks-timesten-vs-oracle-database

NOTE THIS:

One terabyte may not sound like a lot, but because TimesTen also provides columnar compression, it’s equivalent to about five terabytes of addressable storage. Because it’s in main memory, the entire data store can be scanned within five seconds, according to Oracle. Additional Exalytics boxes can be added to increase total system capacity.

http://www.informationweek.com/software/information-management/oracle-strikes-at-sap-hana-with-timesten-database/d/d-id/1102231?

http://www.trivadis.com/uploads/tx_cabagdownloadarea/TimesTen_DOAG_2011_1_.pdf

BTW, I once wrote a time series database which outperformed both TimesTen and Kdb. It supported a SQL-92 syntax. The key to the performance was binding the schema to C++ template based classes, and using memory mapped files. Allowing for on-the-run schema changes is quite expensive, and defining the schema at compile time, data lookups could be retrieved in just a few assembly instructions.http://stackoverflow.com/questions/1593987/alternative-to-the-timesten-in-memory-database

Oracle Advanced Queuing

http://en.wikipedia.org/wiki/Oracle_Advanced_Queuing

http://wedostreams.blogspot.ca/2009/01/oracle-aq-buffered-queues-101.html?m=1

http://www.oracle.com/us/products/middleware/cloud-app-foundation/tuxedo/message-queue/overview/index.html

https://docs.oracle.com/cd/B10501_01/appdev.920/a96587/qintro.htm

http://docs.oracle.com/cd/E11882_01/server.112/e11013/aq_intro.htm#ADQUE0100

NOTE that this is all experimental but so far, I can report it seems pretty decent. As compared to SQL Server 2014 default, it seems not much of a memory hog as compared to SQL Server.

 

Join y FREE newsletter to learn more about this interesting breakthrough

Just in case when things go bad

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Who uses open source Apache Storm for enhanced message queuing?

Who uses open source Apache Storm for enhanced message queuing?

As said on the site:

Storm integrates with any queueing system and any database system. Storm’s spout abstraction makes it easy to integrate a new queuing system

https://storm.apache.org/about/integrates.html

Join my FREE newsletter to learn more about open source projects like this

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Is Redis NOSQL your one stop shop for in memory database with persistance message queuing clustering and TRUE open source

Is Redis NOSQL your one stop shop for in memory database with persistance message queuing clustering and TRUE open source

Man, has this database come a long way which ways always my favourite but the weakneesses seems addressed now

Cluster:

http://redis.io/topics/cluster-tutorial

Persistance:

http://oldblog.antirez.com/post/redis-persistence-demystified.html

Source code looks easy in C which is a  programming language I know well:

http://www.heychinaski.com/blog/2013/10/14/a-look-at-the-redis-source-code/

Potential message queueing:

http://redis.io/commands/rpoplpush

http://restmq.com/ <– this has a good slide deck

I am sure there is more but it sounds like a plan. No longer need Erlang for consideration?

Join my FREE newsletter to learn more about Redis

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Why Erlang is used in HFT environment? Best message queuing server! Goldman Sachs uses it

Why Erlang is used high frequency trading environment?  Best message queuing server! Goldman Sachs uses it

Check out this Erlang binding to one of the best open source mesage queeing http://zeromq.org/bindings:erlang

I even got this thing running on my new Ubuntu Linux environment thanks to VMWare

https://quantlabs.net/blog/2014/11/why-erlang-is-used-high-frequency-trading-environment-best-message-queuing-server-goldman-sachs-uses-it/

Are these good enough reasons to use Oanda as my potential forex trading broker?

This a couple of pointers to start out with but we are not being treated second class due to funding like at other brokers:

Minimum funding required
Local to me as in Toronto
Java API

Let me know what you think

I can confirm you start accessing these APIs with minimal funding like $10. Wow huh?

https://quantlabs.net/blog/2014/11/are-these-good-enough-reasons-to-use-oanda-as-my-potential-forex-trading-broker/

I love this guy!

You got love him: JIM ROGERS – Sell Everything & Run For Your Lives

Nassim who? Taleb what?

https://quantlabs.net/blog/2014/11/you-got-love-him-jim-rogers-sell-everything-run-for-your-lives/

Safely beat the markets with S&P 500 ETF but beat that with automated trading quant and potential HFT

I talked about these links my video :

https://quantlabs.net/blog/2014/11/safely-beat-the-markets-with-sp-500-etf-but-beat-that-with-automated-trading-quant-and-potential-hft/

WTF? All this complexity of learning about automated trading or quant when only 20% of the so called pros beat this benchmark???

OK. I am going to start my count down to Monday DEC 1 for the last day before I double that pricing on the Quant Elite! Interested, learn more here:

http://quantlabs.net/academy/introduction-quant-elite-membership/

Thanks and let me know what you think of above! Many future benefits and recent ones listed here

https://quantlabs.net/blog/2014/11/we-call-this-stacking-the-cool-be-an-elite-quant-before-we-double-the-price-to-learn-how-to-algorithmic-trade-with-potential-hft/

Bryan

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Why Erlang is used high frequency trading environment? Best message queuing server! Goldman Sachs uses it

Why Erlang is used high frequency trading environment?  Best message queuing server! Goldman Sachs uses it

Check out this Erlang binding to one of the best open source mesage queeing http://zeromq.org/bindings:erlang
Check out: http://en.wikipedia.org/wiki/Erlang_%28programming_language%29

Trading

  • Goldman Sachs, high-frequency trading program

Join my FREE newsletter to learn more about this language

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

HFT Secret sauce videos revealed with memory management, MongoDB NOSQL, and ZeroMQ message queuing

Hi there

So what makes a HFT blitzkrieg lightning fast? I just posted this for my QuantLabs.net Premium Members:
“More secret sauce tricks for the ultra fast HFT system I saw in Microsoft .NET C# and Visual C++”
I am not a fan of supervision but I am very ok with this situation. I guess you could say I am be directed by a very technical REAL WORLD quant who has developed an amazingly fast HFT system. You could say I am replicating it at a very low level. Listed below are the latest technologies with video which makes this  thing scream in terms of speed:
1. Youtube video Demo overview of MongoDB NOSQL with Microsoft .NET C# and MonguVue GUI Client
2.  How to install ZeroMQ with C# and .NET binding and example on Microsoft Visual Studio 2012 or 2010
3. Youtube video Demo ZeroMQ with Microsoft .NET C# with Pipeline communication pattern
I hope you recognize the value of this. I just need to integrate all of these with my Matlab Simulink mode that can be code generated. If successful, I am off to the races with a potentially fast system.
The real secrets to these systems is in memory management and the data types chosen. It is not so easy as you think but I am learning it from a world class quant. Want to learn more about the secret sauce of these system?
Thanks Bryan
P.S. I may post the source code once complete but this system will be worth a LOT of money once completed. As a result, there will be a hefty amount to value in my membership.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

ZeroMQ is best for message queuing and parallelism for HFT and quant development? C++, LUA, Python Youtube video demo

ZeroMQ is best for message queuing and parallelism for HFT and quant development? C++, LUA, Python Youtube video demo

I have been introduced to Zero MQ when it came out a few years ago. I was excited when I saw what it could. I am now investigating this as an option for new upcoming HFT platform. You might want to take a look at this video to see what I mean.

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Message Queuing Framework for quant development

Message Queuing Framework for quant development

In order to reduce message’s latency and guarantee high availability at the same time, I consider different to use message queuing framework, such as OpenAMQ and Apache Qpid. Both are based on the AMQP standard.
Please let me know what do you think about them and/or if you adopt other frameworks/solutions.

Apache Qpid was in incubation for a while. I haven’t played with it anyway. I am doing some independent exercising with ActiveMQ, but I don’t expect high performance from it either http://bhavin.directi.com/rabbitmq-vs-apache-activemq-vs-apache-qpid/

We implemented a pub/sub based system for controlling price limits and settlement prices once. We started with a pub/sub server developed by the french branch (Euronext), but it failed to cope with the message volume (protocol parsing and decoding, ~10k-15k msg/sec during the pick time), so the server side team ended up writing one from scratch (C plane structures with some throttling so that GUI side won’t crash). But again, it depends on requirements. These days I would certainly consider DDShttp://drdobbs.com/architecture-and-design/222900238

I’ve been using JBoss HornetQ, really fast! try ithttp://www.jboss.org/hornetq

By far, the fastest messaging is 29 West’s LBM. It uses multicast groups as a routing mechanism, so the routers deal with message traffic moving things across subnets very efficiently. They claim some 1.4 million messages per second sustained, and have full blown queueing built on top of the framework, if you want it. Give durable subscriptions, pub/sub, request/reply, etc.

Used by the NY Stock Exchange as their messaging backbone, and sold as part of the NYSE market data software suite. Expensive, but ultra low latency.

http://en.wikipedia.org/wiki/29West

 

Another good option is ZeroMQ [http://www.zeromq.org/] (Zero as in “No Broker”) . There is very broad language coverage of its api.

It too is very fast, I use it as a messaging wrapper over the sockets library.

Below is point summary from the website:

Ø  The socket library that acts as a concurrency framework.
Ø  Faster than TCP, for clustered products and supercomputing.
Ø  Carries messages across inproc, IPC, TCP, and multicast.
Ø  Connect N-to-N via fanout, pubsub, pipeline, request-reply.
Ø  Asynch I/O for scalable multicore message-passing apps.
Ø  Large and active open source community.
Ø  20+ languages including C, C++, Java, .NET, Python.
Ø  Most OSes including Linux, Windows, OS X.

second that

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!