Tag Archives: Databases

12 new databases from Quandl FREE Data

12 new databases from Quandl FREE Data

We have three new databases for institutional and professional users:

Dry Bulk Freight Forwards Rates

We now offer dry bulk freight forward rates for the major vessel classes. Published by MP Maritime, this database is powered by feeds from FIS sourced continuously from traders all over the world.  This data is valuable, exceptionally difficult to get and the daily update time of 18.45 GMT means it is also extremely timely.  History to 2008.

US Corporate Bonds

Quandl now provides fair-value EOD prices, yields, spreads, DV01s and other metrics for over 13,000 corporate bonds sourced directly from FINRA TRACE by our newest premium vendor, DelphX.  9 years of history means this database can be used as a definitive reference for research or auditing.  The timely 4pm ET update schedule gives institutions an MTM solution at a price 5 to 10 times lower than other providers.  You have trial access»
Point-in-time Earnings Estimates

Quantopian contestants and other alpha-seeking quants take note:  Zack’s newest database offers point-in-time earnings estimates for 5000 companies on a quarter-by-quarter basis back to 2009.  Whatever signals are buried in consensus earnings estimates and their trends can be gleaned from this unbiased historical database.  You have trial access»

Open Data News We have 9 brand new, beautifully organized and completely free cross-country databases from the United Nations Statistical Division, covering Commodity Production, Commodity Trade, Energy, Environment, Food & Agriculture, Gender Statistics, Pollution, Information + Communication Technology, and World Tourism.  We will be adding more databases from UNSD soon.

Around the Web:

– Denes Csala built an interactive data plotter using Quandl and NVD3.
– Aakash Gupta built a Shiny app for technical charting of the Indian stock market.
– Derek Braid built a Meteor app with working ticker search, for the Canadian market.
– Ari Lamstein used Shiny to build boxplots, maps and graphs of violent crime in the US.
– Lekker Logic created and easy-to-use commodity charter using Charts.js and PHP.
– Quinn-Curtis have built a powerful and comprehensive charting package with Quandl as a core data source.
If you’ve built something interesting using Quandl data, please do share!
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Questions for databases tonight: NOSQL vs SQL database smackdown, which one for what?

Questions for databases tonight: NOSQL vs SQL database smackdown, which one for what?

All related about high speed access, throughput for housing tick/bar market time series data in very high volume.

SQL:

Who uses Oracle for high speed live trading and data store? Feedback??

Who uses SQL Server for high speed live trading and data store? Feedback??

Anyone ever do apple to apple comparison between SQL Server and Oracle?

Upper limits of performance hit with MYSQL or Prestgres?

NOSQL:

Who used Cassandara? Anyone still using it?

Overall perspectives of the NOSQL solution?

NOSQL:

Anyone use Redis? Try out the clustering or message queuing?

Anyone see limitation on MongoDB insert throughput? Does it cluster at higher transaction volume? Does it corrupt data?

Anyone ever look other solutions like Aerospike or Cassandra? Looking for same info on the above.

Anyone have any luck with the above limitation NOSQL options:

Roll your own solution:

Which language? Which design patterns? How do you handle message queueing? Clustering? Fail over? Redundancy?]

Anyone ever work with flat files? What language? Any compression algo techniques? Performance issues? How do you manage all of them?

Ever look at a solution like TEA Files in C#?

Performance better overall?

Remember that there is no public playback for this event.  You will only be able to get access here

Join my FREE newsletter to learn more about these events

LIVE Event: NOSQL vs SQL database smackdown, which one for what?

Monday, Dec 15, 2014, 7:00 PM

Oline via GotoMeeting
GotoMeeting Toronto, ON

11 Members Went

Online Meetup LIVE Event: NOSQL vs SQL database smackdown, which one for what?There are so many databases usedbut what for what reason. I really want to investigate this with others online. If you are interested in our poll on this on popularity, please, visit:http://quantlabs.net/surveys/tick-db/This should also cover the experience with pros a…

Check out this Meetup →

LIVE Event: NOSQL vs SQL database smackdown, which one for what?

Monday, Dec 15, 2014, 7:00 PM

GotoMeeting Webinar online
GotoMeeting Webinar online Toronto, ON

23 Researching Traders Went

Online Meetup LIVE Event: NOSQL vs SQL database smackdown, which one for what?There are so many databases usedbut what for what reason. I really want to investigate this with others online. If you are interested in our poll on this on popularity, please, visit:http://quantlabs.net/surveys/tick-db/This should also cover the experience with pros a…

Check out this Meetup →

LIVE Event: NOSQL vs SQL database smackdown, which one for what?

Monday, Dec 15, 2014, 7:00 PM

13 Members Went

Check out this Meetup →

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How does Aerospike compare vs Redis and other NOSQL databases?

How does Aerospike compare vs Redis and other NOSQL databases?

From their business manager:

http://stackoverflow.com/questions/24482337/how-is-aerospike-different-from-other-key-value-nosql-databases

Thanks to him for sending

Join my FREE newsletter if you want to learn more about NOSQL solutions

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Here are some further thoughts of comparison of Redis vs Erlang NOSQL databases: Couchbase Mnesia Riak

Here are some further thoughts of comparison of Redis vs Erlang NOSQL databases: Couchbase Mnesia Riak

These are even more confusing but helps for resolution here:

Couchbase vs Redis

Couchbase master to master replication unique and also has no locking concepts!

http://db-engines.com/en/system/Couchbase%3BRedis

SImple for Redis:

For example, if you were working on a high performance algorithmic trading and you were pulling ticker prices out of a firehose and needing to store them at an insane rate so they could be processed, Redis is exactly the kind of datastore you would want to turn to for that — definitely not Mongo, CouchDB or MySQL.

http://rick-hightower.blogspot.ca/2014/04/well-put-when-should-you-use-mongodb-vs.html

This is also outdated

 

Booto Couchbase:

Memcached vs. Couchbase:
We compared the performance of a standard Memcached (1.4.x) server with a Couchbase server (not version 2) and saw ~40% performance degradation with Couchbase.  This is mainly because Couchbase doesn’t not serve all the items from in-memory.

Yeh to Redisl

But Memcached can also run over a multi-threaded architecture, so unless you shard your Redis dataset, Memcached should have performance advantages here.
That said, Redis supports pipelining, which is ~x5 faster than without pipelining, and Memcached doesn’t support pipelining capability.

Memory is cheap for this via a local server:

The one downside of Redis is the memory overhead. You need 2X RAM to store X data. Memcached is a little more efficient & transparent with memory organization – you can examine the slabs, etc.

http://www.quora.com/Performance/Does-Redis-or-Couchbase-noSQL-have-total-higher-throughput-when-used-as-a-memcached-replacement

This article could resell you on Couchbase:

http://www.computing.co.uk/ctg/news/2330148/viber-explains-why-it-ditched-mongodb-and-redis-in-favour-of-couchbase

It sounds like a NO to Mnesia:

  • Erlang only interface
  • Tables limited to 2GB
  • Deadlock prevention scales poorly
  • Network partitions are not automatically handled, must recombine tables automatically

http://nosql.mypopescu.com/post/711855931/all-erlang-riak-and-mnesia

Confused? Maybe I should start with Couchbase for the Erlang since it SHOULD be easier to integrate. I also like the lockless capability and they do cover all the major language bindings I want.

Join my FREE newsletter to learn more about which way I got

How to install Redis into Erlang

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Redis could be better choice for tradng over MongoDB NOSQL if you using Erlang but they got databases to (call me extra confused)

Redis could be better choice for tradng over MongoDB NOSQL if you using Erlang but they got databases to (call me extra confused)

The weakness could be overcome as you run multiple instances on this for Redis
http://redis.io/topics/partitioning

The deal breaker for Redis over Mongo:

Maybe Redis is better due to MongoDB Erlang drivers are community driven

http://docs.mongodb.org/ecosystem/drivers/

Erlang database options but never tested:

Why big companies use Mnesia instead of using Riak or CouchDB – Stack Overflow

http://stackoverflow.com/questions/23180484/why-big-companies-use-mnesia-instead-of-using-riak-or-couchdb

3 database for Erlang Mnesia Riak and CouchDB
https://www.erlang-solutions.com/products/riak-nosql-database

(looking for opinions here)

historical trend of the popularity ranking of database management systems

http://db-engines.com/en/ranking_trend

Join my FREE newsletter to learn more about why way I go with a database

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Vitesse: Using LLVM To Speed Up Databases like my PostGres SQL

Vitesse: Using LLVM To Speed Up Databases like my PostGres SQL

This was sent in by Skype user so thanks to him for sending

Vitesse: Using LLVM To Speed Up Databases

I call my solution SQL Server 2014 which is fast and professional for future use.

http://www.phoronix.com/scan.php?page=news_item&px=MTgxNzI

Join my FREE newsletter to learn more how to implement this

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

3 hour video Q&A Meetup with London Quant on databases machine learning and loads of other tech talk from an awesome trader

 

3 hour video Q&A Meetup with London Quant on databases machine learning and loads of other tech talk from an awesome trader

NOTE TO ALL: It has been strongly recommneded to make these future session private for only paying members only.

Wow This was a 3 hour event where lots were anwered. We covered machine learning, database, lots of tech, and so on. Start learning from the best now!

It was said how we do surpass this one a it was easily the best yet!

Join my FREE newsletter to learn about other events online

Important links from the discussion:

http://discretelogics.com/teafiles/

http://www.amazon.ca/Handbook-Neuroevolution-Through-Erlang-Gene/dp/1461444624

http://www.nanex.net/aqck2/4680/20140929_10.06.48.000_1000ms_eGOOGL.0.gif

http://www.springer.com/computer/swe/book/978-1-4614-4462-6

 

 Here is the entire exchange thread of Meeutup:

Bryan Downing (to All – Entire Audience): hello everyone

Bryan Downing (to All – Entire Audience): hey all

Bryan Downing (to All – Entire Audience): how is everyone doing tonite or today?

John Ruiru Munge (to All – Entire Audience): Yes I can hear you

Bryan Downing (to All – Entire Audience): http://www.gnuplot.info/

Edwol (to All – Entire Audience): no

Edwol (to All – Entire Audience): I do some python, occasionally VB

Gigi (to All – Entire Audience): https://www.youtube.com/watch?v=hcRUqBm9GFI

John Ruiru Munge (to All – Entire Audience): thanks

Jeff (to All – Entire Audience): Hi all.

Jeff (to All – Entire Audience): Great how are you. Thanks

Ven (to All – Entire Audience): Hello Everyone

Geoff (to All – Entire Audience): Hello everyone

Edwol (to All – Entire Audience): Hi

Jeff (to All – Entire Audience): Sorry Gigi, can you please repeat what you said about the Interactive brokers connection?

George (to All – Entire Audience): Hello Everyone

George (to All – Entire Audience): What time is it inLondon?

Bryan Downing (to All – Entire Audience): hey

George (to All – Entire Audience): Can you give an introduction of London Quant?

Edwol (to All – Entire Audience): y

Geoff (to All – Entire Audience): I use Python

John Ruiru Munge (to All – Entire Audience): basic knowledge of python at the moment

Jeff (to All – Entire Audience): I use python

James Hirschorn (to All – Entire Audience): neve used it

Ven (to All – Entire Audience): Java . basic python

sysuser (to All – Entire Audience): I cant hear anything

sysuser (to All – Entire Audience): Is all ok ?

Edwol (to All – Entire Audience): y

John Ruiru Munge (to All – Entire Audience): yes

James Hirschorn (to All – Entire Audience): fine

George (to All – Entire Audience): hear you okay

Edwol (to All – Entire Audience): y

Jeff (to All – Entire Audience): n

George (to All – Entire Audience): yes

Sholom (to All – Entire Audience): yes

James Hirschorn (to All – Entire Audience): not yet

Geoff (to All – Entire Audience): yes

Ven (to All – Entire Audience): not yet

sysuser (to All – Entire Audience): I cant hear anything

Geoff (to All – Entire Audience): yes

John Ruiru Munge (to All – Entire Audience): N

Edwol (to All – Entire Audience): n

George (to All – Entire Audience): yes

sysuser (to All – Entire Audience): Please sugest if anyone hearing anything

Karen Tursunov (to All – Entire Audience): no

Ven (to All – Entire Audience): syuser- try connecting again.

George (to All – Entire Audience): no

Edwol (to All – Entire Audience): n

Karen Tursunov (to All – Entire Audience): no

Geoff (to All – Entire Audience): yes,

John Ruiru Munge (to All – Entire Audience): quantopian only

Sholom (to All – Entire Audience): no

John Ruiru Munge (to All – Entire Audience): what are your backgrounds

Shane (to All – Entire Audience): any automated system design that outperforms others? how frequently do you have to change strategies? what is the lifetime of a strategy?

Geoff (to All – Entire Audience): Check 12 strategies once a month

James Hirschorn (to All – Entire Audience): Gigi: Can you please type the name of the legit binary options broker? Thanks.

Gigi (to All – Entire Audience): Dukascopy, maybe IG

Geoff (to All – Entire Audience): optimize and walk forward every 30 minues

Shane (to All – Entire Audience): cool, thanks

Edwol (to All – Entire Audience): how much of a role if any does machine learning play in your systems ?

Jeff (to All – Entire Audience): Are your non-linear elastic timeframes related to fibonacci numbers?

Alex (to All – Entire Audience): They’re 10,000 a month starting with cheaper services, and up to 30k a month for bloomberg/reuters digitized news streams.

Steve Eberbach (to All – Entire Audience): I would call elastic time windows “event or conditionally closed- opened bars”. What events or conditions would you use to define time-elastic sampling intervals in streaming sequential data?

Shane (to All – Entire Audience): can you describe your systems at a high level?

Alex (to All – Entire Audience): up to 30k a month for bloomberg/reuters digitzed news

Alex (to All – Entire Audience): Are you using separate systems to process, backtest and analyze OLTP (in memory) and OLAP (historical data)?

Alex (to All – Entire Audience): database technology, in memory database, separate for historical data?

Alex (to All – Entire Audience): 2 different systems or just 1 for in memory?

Shane (to All – Entire Audience): can we get an intro /overview to start?

Alex (to All – Entire Audience): Could your system adapt to a 2008-level event, have you backtested drawdowns for that yr?

George (to All – Entire Audience): Automatic trading systems affect the order flow of the products it is trading. In principle, order flow change can effect how other traders trade around it. Is there a limit on how long one can can use an automatic trading system before it become ineffective? ie no longer generates a profit.

Alex (to All – Entire Audience): so you’re talking about mean-reverting strategies on the eurusd?

Alex (to All – Entire Audience): do you consolidate bar data from ticks or do do you make second or minute based queries to the database?

Alex (to All – Entire Audience): what do you think of newsql, or do you use your own nosql database with a c++ api?

Bryan Downing (to All – Entire Audience): check out teafiles

Shane (to All – Entire Audience): what is your average trade length in wall clock seconds? average price differential (pennies? ticks?

Bryan Downing (to All – Entire Audience): http://discretelogics.com/teafiles/

Sholom (to All – Entire Audience): How do you use Monte Carlo to test a non-parametric method?

Alex (to All – Entire Audience): what do you think about open-source in-memory databases used in conjunction with something like hbase/hadoop for historical data?

Alex (to All – Entire Audience): is your system lock-free for multiple threads?

Alex (to All – Entire Audience): so you don’t like matlab, r, or python for backtesting? You prefer .net for backtesting, and c++ for production?

George (to All – Entire Audience): Do you encript data your data?

Sholom (to All – Entire Audience): What do you use to estimate the current span of memory in the market (the set of possible market behaviors that are already factored into the price) ?

George (to All – Entire Audience): What role does volatility play in your trading system

Shane (to All – Entire Audience): how do you diivide your research time? improving technology implementation vs stategy development? what % time on each? or is there some other activity worth more time?

Alex (to All – Entire Audience): so it works like a mean-reverting seasonal/temporal strategy based on correlation optimized by a genetic algorithm?

 

 

Alex (to All – Entire Audience): you don’t like GA, you prefer decision trees and NNs?

Edwol (to All – Entire Audience): So you use SVM for patterns?

Alex (to All – Entire Audience): SOMs are technically NNs, though, no?

Alex (to All – Entire Audience): so you don’t like matlab, r, or python for backtesting? You prefer .net for backtesting, and c++ for production?

 

 

 

Shane (to All – Entire Audience): what is the biggest challenge you face now? improving models? liquidity? technology? other?

Sholom (to All – Entire Audience): when creating your own volatility function what are the basic input parameters you must consider?

Sholom (to All – Entire Audience): I propose the following: ?(? , ?1, ?2)

 

>>Where ? = S/A and A is the psychological price threshold level discovered using an empirical/non-parametric method.

>> ?1 = The price path to get to the current price S (specific price movement plus price velocity)

>> ?2 = The current span of memory in the market (the set of possible market behaviors that are already factored into the price)

Alex (to All – Entire Audience): what

Alex (to All – Entire Audience): the difference between GA and GP in your words?

Jeff (to All – Entire Audience): What is the minimum educational barrier for entry into HFT, do you need PhD in STEM related subject to succeed in HFT?

Steve Eberbach (to All – Entire Audience): On difference between genetic programming versus optimization: With genetic programming you do not consider the model-evolving program’s setting of the coefficients of terms as ” optimization” as you would when fitting optimim coefficients to terms for “pre-assumed equations” with genetic algorithm.?

Bryan Downing (to All – Entire Audience): https://bigml.com/

Shane (to All – Entire Audience): how often tdo you produce new models? what approach/process do you use to develop a model?

Edwol (to All – Entire Audience): do you specify the inputs like mutatiion % crossover, tournament size or is it random in your GP ?

Alex (to All – Entire Audience): related to edwol, so how do you deal with low-level selection, mutation, fitness, etc. levels?

Jeff (to All – Entire Audience): Have you tried combining evolutionary algorithms (GAs) with machine learning to find the optimum inputs to neural networks and/or any other model(s)?

Geoff (to All – Entire Audience): During the london day, how many periods are your non linear elastic periods broken into for the EUR

Alex (to All – Entire Audience): ga inside gp

Steve Eberbach (to All – Entire Audience): Have you tried the GP software called Discipulus?

Shane (to All – Entire Audience): I have to drop off, thanksvery much for your time

Alex (to All – Entire Audience): So GP just selects the proper GA from multiple simulated GAs?

Alex (to All – Entire Audience): basically us and asia time zones

Bryan Downing (to All – Entire Audience): any more questions?

Alex (to All – Entire Audience): GP has code generation?

Jeff (to All – Entire Audience): GP writes code in which languages? Highlevel or lowlevel like assembler?

George (to All – Entire Audience): I beleive that your system uses a some form of a machine learning strategy. Does your system make mistakes? If so, do you have a safeguards to prevent errors that could lead to losses?

Alex (to All – Entire Audience): so you coded your own custom GP library, there are many various software libraries that do GA, but I haven’t seen the GP you’re talking about.

Alex (to All – Entire Audience): so you’re just using windows/.net for gui passed onto windows using a messaging middleware, but it’s replacable, for instance you could also do web gui or android app with the middleware communication?

Bryan Downing (to All – Entire Audience): more questions ?

Jeff (to All – Entire Audience): Can I ask how profitable your systems are? (%)

Is your trading history public anywhere?

Edwol (to All – Entire Audience): How do you verify the veracity of your GP – eg confusion matrix, ROC …

Alex (to All – Entire Audience): but you continue to use windows for guis? what do you think about the qt library?

Edwol (to All – Entire Audience): great

Bryan Downing (to All – Entire Audience): more questions?

Bryan Downing (to All – Entire Audience): more comments?

Bryan Downing (to All – Entire Audience): wpf and xaml folks

Bryan Downing (to All – Entire Audience): more questions or comments ?

Jeff (to All – Entire Audience): Can HFT systems be designed for GNULinux?

Alex (to All – Entire Audience): what do you think of religious .net believers or for programming languages in general? .net/windows still is limited by the os vs. linux

Bryan Downing (to All – Entire Audience): any other questions or comments?

Alex (to All – Entire Audience): so why do you hate the jvm?, mono is garbage for high-performance systems, I only see it being used for niche, code generation on android for instance, or for some video games.

Bryan Downing (to All – Entire Audience): questions from the newbies?

Bryan Downing (to All – Entire Audience): comments ?

Jeff (to All – Entire Audience): Newbie here: Your systems are deployed to Linux or Windows servers?

George (to All – Entire Audience): Is trading volume used as aan imput for system’s decision to make a trade

Alex (to All – Entire Audience): so what do you think of erlang’s approach to multhreading?

Alex (to All – Entire Audience): how decent is fx volume data from lmax? accurate?

Jeff (to All – Entire Audience): related to Alex’s question above, any Lisp or Clojure used?

Ven (to All – Entire Audience): what is the ping time between your system and exchange on average

Jeff (to All – Entire Audience): gr8 question Ven

Alex (to All – Entire Audience): so you go from from c++/qt to .net, but you ignore the middleman, java, why?

Alex (to All – Entire Audience): do you connect to muliple LPs through lmax, or do you have independence t connections?

Jeff (to All – Entire Audience): Are you co-located next to the exchange or in a datacenter like LD4

Alex (to All – Entire Audience): so do citi, jpm, etc. in addition to lmax?

Alex (to All – Entire Audience): do you use a prime of prime fx brokerage?

George (to All – Entire Audience): Does the majority of your system profit’scome from retail traders or instituanal traders

James Hirschorn (to All – Entire Audience): Have to go. Thanks!

Alex (to All – Entire Audience): are there any financial sites you frequent, besides your bloomberg terminal?

Ven (to All – Entire Audience): what part of the system do we need to fine tune improve the ping time – strategy code, broker api, hardware, network IO or everything

Alex (to All – Entire Audience): so no mnra or ssrn? You don’t buy the academia?

Jeff (to All – Entire Audience): where can I get the book?

George (to All – Entire Audience): Trading is a zero sum game. One makes profits from another traders losses. Are the majority your profits profrits from retail traders (eg Joe trader who day traders for a living) or from instituational traders (eg traders who trade for Banks,)?

Jeff (to All – Entire Audience): sorry, what’s the title again?

Alex (to All – Entire Audience): so you prefer direct engineering/machine learn journals to financial journals/papers?

Bryan Downing (to All – Entire Audience): http://www.amazon.ca/Handbook-Neuroevolution-Through-Erlang-Gene/dp/1461444624

Sholom (to All – Entire Audience): http://books.google.com/books?id=MgNmy8PIeyQC&printsec=frontcover#v=onepage&q&f=false

Jeff (to All – Entire Audience): sorry my phone call dropped.   But I hope the video of this will be posted.

Jeff (to All – Entire Audience): gr8 thanks Sholom

Daniel Howard (Siesta Key, FL) (to All – Entire Audience): http://uploaded.net/file/xh3dtyuf/Handbook.of.Neuroevolution.Through.Erlang.pdf

Jeff (to All – Entire Audience): gr8 Daniel, tnx

Sholom (to All – Entire Audience): https://github.com/CorticalComputer/Book_NeuroevolutionThroughErlang

Alex (to All – Entire Audience): what do you think of hedge funds, they seem to not outperform the market, like the efficient market theory would say. Would you say that’s because people who can make money through prop trading, do not need to lose ‘other people’s money’?

Alex (to All – Entire Audience): do you have any investment recommendations for long-term holdings? Do you hold index etfs, cds, etc?

George (to All – Entire Audience): London Quant: Thanks for your discussion.

Alex (to All – Entire Audience): so you’re a pure trader and otherwise in cash?

Daniel Howard (Siesta Key, FL) (to All – Entire Audience): Thanks, LQ

George (to All – Entire Audience): Thanks Bryan

Alex (to All – Entire Audience): for long-term holdings, otherwise short-term fx speculation

Geoff (to All – Entire Audience): Thank you very much from SF

Sholom (to All – Entire Audience): Thank you very much!

Edwol (to All – Entire Audience): Thanks LQ

Ven (to All – Entire Audience): Thanks London Quant. Great learning for me

Daniel Howard (Siesta Key, FL) (to All – Entire Audience): Night all

Jeff (to All – Entire Audience): Awesome meeting, thanks Bryan, thanks LondonQuant

Bryan Downing (to All – Entire Audience): no problems jeff

Alex (to All – Entire Audience): do you load data into memory for backtesting? How much memory do you have/recommend for a single node to handle an in-memory db?

Alex (to All – Entire Audience): do you use centos? base-metal no virtualization?

Jeff (to All – Entire Audience): can HFT methods be applied to crypto currency markets?   Do you believe HFT enter crypto currency trading in future?

Rahul (to All – Entire Audience): what kind of other systems do you use to form strategies

Jeff (to All – Entire Audience): Can Joe Trader enter HFT?

Jeff (to All – Entire Audience): btw: thank you Sholom for the github link

Sholom (to All – Entire Audience): You’re welcome Jeff.

Alex (to All – Entire Audience): so the way your db is designed, it can’t work with unstructured data, only time-series data?

Jeff (to All – Entire Audience): thanks Gigi

Rahul (to All – Entire Audience): R or matlab?

Alex (to All – Entire Audience): so you use python for scripting on linux for maintenance of packages, etc. but not for critical trading?

Sholom (to All – Entire Audience): yeah Python has tons of bugs

Alex (to All – Entire Audience): you deleted this: so the way your db is designed, it can’t work with unstructured data, only time-series data?

 

 

 

Bryan Downing (to Jeff): any more questions?

Bryan Downing (to All – Entire Audience): any more questions?

Bryan Downing (to All – Entire Audience): comments?

Alex (to All – Entire Audience): do you use kde on linux? what’s your preferred de?

Alex (to All – Entire Audience): so you build c++ on eclipse for your system or are you using c++/qt?

Jeff (to All – Entire Audience): Do you trade independently or for a bank/fund?

If independent, how do you structure your corp/company for tax reasons?

Alex (to All – Entire Audience): how much hair have you pulled out over c++ and dealing with pointers for your system?

Alex (to All – Entire Audience): do you have semi-manual execution, more grey-box than black box? watching bloomberg terminal for events, etc.?

Alex (to All – Entire Audience): so you have execution traders in addition to programmers/engineering experts?

Geoff (to All – Entire Audience): No

Alex (to All – Entire Audience): are you spaniard?

Jeff (to All – Entire Audience): no

Jeff (to All – Entire Audience): Alex: Israel or Italy

Bryan Downing (to All – Entire Audience): does background really matter?

Alex (to All – Entire Audience): Genuine question: you’re good at reading accents, jeff?

Jeff (to All – Entire Audience): Alex: no..   so so, not good, not bad

Jeff (to All – Entire Audience): depending on which language

Bryan Downing (to All – Entire Audience): let’s focus on trading folks

Geoff (to All – Entire Audience): I agree

Jeff (to All – Entire Audience): Can I follow your lifestyle on Istagram or Facebook?

Alex (to All – Entire Audience): do you use basic visualizations like line charts for pnl, etc. or do you use more advanged graphics like heatmaps, etc.?

Bryan Downing (to All – Entire Audience): lightning charts

Gigi (to All – Entire Audience): http://www.nanex.net/aqck2/4680/20140929_10.06.48.000_1000ms_eGOOGL.0.gif

Gigi (to All – Entire Audience): http://www.nanex.net/aqck2/4680/20140929_08.56.20.000_20000ms_eGOOGL.1.gif

Alex (to All – Entire Audience): so you’re a full-time in house shop for everything from machine learning libraries, to the database to charts to even custom linux distribution, how much man years would you estimate that it would take to build it?

Rahul (to All – Entire Audience): HFT can lead to market crash sometimes?

Jeff (to All – Entire Audience): What software management tools do you use, if any. SVN/Suubversion, Git, etc.?

Alex (to All – Entire Audience): what do you think of news sentiment and analysis?

Alex (to All – Entire Audience): but you watch bloomberg, isn’t that hypocritical?

Alex (to All – Entire Audience): terminal, vs. using the bloomberg feed

Rahul (to All – Entire Audience): how do you manage trades at events?

Rahul (to All – Entire Audience): dividend and merger

Alex (to All – Entire Audience): exactly, you’re not quantifying sentiment, you’re still using discretionary bloomberg terminal, not using the machine readable feed and performing sentiment analysis on it

Alex (to All – Entire Audience): what do you think of the libor scandal?, many institutions are now banning bloomberg chat use for their traders via compliance

Rahul (to All – Entire Audience): @ Bryan: Keep on sharing your knowledge

Jeff (to All – Entire Audience): How do you manage your concentration when developing/programming your systems, do you outsource any tasks or build solo?

Alex (to All – Entire Audience): are you colocating your own hardware at ny4?

Jeff (to All – Entire Audience): Do you use Agile or any other methodologies like scrum.

Do you unit test your code?

Alex (to All – Entire Audience): so are you using any x86 asm, or is it pure c++, so it could run on other archs eventually?

Alex (to All – Entire Audience): you prefer gcc or clang?

Alex (to All – Entire Audience): do you use sharpe ratios, information ratios, etc. or do you focus on pure pnl per trade?

Jeff (to All – Entire Audience): Can you apply scalability of the web/mobile development methodologies for financial markets / HFT?

Alex (to All – Entire Audience): so you’re focusing specifically on the market microstructure of the euro, and if the euro broke-up it wouldn’t work?

Alex (to All – Entire Audience): so it’s semi-directionary, and you have your own EMS?

Alex (to All – Entire Audience): discretionary

Alex (to All – Entire Audience): so the ai engine is more of an advisor agent?

Rahul (to All – Entire Audience): some good strategies that you would like to share?

Jeff (to All – Entire Audience): If you have children teenagers, are your children any of the rich kids of instagram showing lambos, ferraris, yachts, and private jets?

Alex (to All – Entire Audience): what do you think of mandelbrot’s theories?

Jeff (to All – Entire Audience): Can you use atronomical data from as inputs to your neural nets?

Jeff (to All – Entire Audience): 🙂

Bryan Downing (to All – Entire Audience): ha ha

Alex (to All – Entire Audience): what do you think of the hurst exponent or markov models, what do you use to detect regimes or is this embedded in the GP?

Rahul (to All – Entire Audience): what do you think about future of algo trading? do algo traders reallly make money all time? is it really effective?

Alex (to All – Entire Audience): do you use boost? Did you design your own fix engine?

Geoff (to All – Entire Audience): Thanks Bryan and London Quant from SF

Jeff (to All – Entire Audience): Awesome talks Bryan. thanks Bryan, Gigi, LondonQuant and everyone making this possible.

Rahul (to All – Entire Audience): Thanks to all

Sholom (to All – Entire Audience): Best one yet! Thanks LQ!

Edwol (to All – Entire Audience): This was awesome – thanks to all who participated

Alex (to All – Entire Audience): talk about together in another webinar after feedback exchange

Jeff (to All – Entire Audience): How long have these talks been going on?

Jeff (to All – Entire Audience): I mean the meetups

Jeff (to All – Entire Audience): Is it all online?   I neeed to catch up

Steve Eberbach (to All – Entire Audience): Bryan! This was a good pow wow. I had to take my kid to the doctor, and now I’m back! Hope you have recorded the audio of what I missed!

Geoff (to All – Entire Audience): Bye all

Jeff (to All – Entire Audience): Thanks, bye all.

Steve Eberbach (to All – Entire Audience): Bye!

Edwol (to All – Entire Audience): bye

Rahul (to All – Entire Audience): bye

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Are key databases like FlockDB from Twitter Goldman Sachs SecDB or TEAFiles key to your high speed trading system aka HFT ?

Check out these projects which massively helped out these operations:

Twiiter:  http://en.wikipedia.org/wiki/FlockDB https://github.com/twitter/flockdb

This is respected database that couild be used fpr trading but don’t count on it

This could be a secret weapon for Goldman Sachs:

Is success of Goldman Sachs in HFT and quant development with Slang/SecDB no different than Erlang/Oracle Berkeley DB?

I little birdie told me they moved to Ruby on Rails but I would asume that is the front but still with SecDB on the back end.

Where do you think high speed kdb came from? Uh. Morgan Stanley: http://en.wikipedia.org/wiki/K_%28programming_language%29

In essenvce Redis will not cut it where a homegrown version is need with graph and NOSQL with a query language is needed. Being C and cross platform helps.

Join my FREE newsletter if you want to learn about this database thing

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Webinar on using Matlab Production Server for Financial Application with financial analytics into Excel SAS databases, and Hadoop big data , and web applications

 

Webinar on using Matlab Production Server for Financial Application with financial analytics into Excel SAS databases, and Hadoop big data , and web applications

Here is the best sample use for this powerful product:

http://www.mathworks.com/videos/matlab-production-server-for-financial-applications-81937.html

Join our FREE newsletter to see how this product can be implemented 

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

TONIGHT SQL Databases with MYSQL SQL Server NOSQL MongoDB with Automated Trading Systems webinar with quant and HFT potential

HI there

Here are the details for tonite’s Workshop event on :
I have my last LIVE workshop event for 2013 for tonight at 7 PM Eastern Time:

Quant Members Workshop schedule for Trading Infrastructure: Databases like MYSQL or SQL Server for automated trading systems – See more at: https://quantlabs.net/blog/2013/11/quant-members-workshop-schedule-for-trading-infrastructure-databases-like-mysql-or-sql-server-for-automated-trading-systems/#sthash.3huP3TVZ.dpuf

 

Login details at:

….

Hope you can make it out!

Want access? Get it here!

Need more info? Get it here?

Join our FREE newsletter to learn more about topics like this
Bryan

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!