Tag Archives: Spark

Google searches lower for Bitcoin while volatility could spark full blown crash

Here are two of my latest video with links:

 

http://quantlabs.net/academy/google-searches-lower-for-bitcoin-while-volatility-could-spark-full-blown-crash/

 

http://quantlabs.net/academy/djia-suffered-last-week-while-germany-confirms-4th-surplus-economy/

 

You will notice that these are behind a paywall. You can join this to get access.

Another way is listening to the videos below which are audio podcast version below.

 

Thanks for your input,

 

P.S. New videos are coming to complete the cycle of this new passive strategy I have been working on for the past few weeks. This may start tomorrow.

 

Input caption text here. Use the block’s Settings tab to change the caption position and set other styles.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Spark as a Real-Time Web Service PDF

Spark as a Real-Time Web Service PDF

This does 700 updates per second so should I be impressed?

https://www.slideshare.net/mobile/EvanChan2/700-updatable-queries-per-second-spark-as-a-realtime-web-service

For me Redis is super impressive!! http://redis.io/topics/benchmarks

redis-benchmark -t set -r 100000 -n 1000000
====== SET ======
  1000000 requests completed in 13.86 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

99.76% `<=` 1 milliseconds
99.98% `<=` 2 milliseconds
100.00% `<=` 3 milliseconds
100.00% `<=` 3 milliseconds
72144.87 requests per second

$ redis-cli dbsize
(integer) 99993

redis-benchmark -n 1000000 -t set,get -P 16 -q
SET: 403063.28 requests per second
GET: 508388.41 requests per second

Join my FREE newsletter to see if Spark is earth shattering for Automated trading

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Is Scala and Spark the secret trader delight

Is Scala and Spark the secret trader delight

More shops could be using it secret but I don’t like Java nor the JVM

View at Medium.com

Join my FREE newsletter to learn if I use Scala for automated trading

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Implement machine learning distributed deep learning network on Spark

Implement machine learning distributed deep learning network on Spark

Would this be fairly fast since it is supposed to in memory

From Super Facebook fan Nuno so thanks to him

http://www.datasciencecentral.com/profiles/blogs/implementing-a-distributed-deep-learning-network-over-spark

Join my FREE newsletter to see if this ever gets implemented in my automated trading

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

NOSQL Redis makes open source Spark faster

NOSQL Redis makes open source Spark faster

Thanks to Super Fan Nuno for sending this over

Join my FREE newsletter to see how I use NOSQL Redis for my algo trading infrastructure 

Redis Connector Aims to Boost Spark Performance

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Use open source Spark R to power shiny apps

Use open source Spark R to power shiny apps

I am unsure of the performance here of what I offer but Spark is popular

http://blog.sparkiq-labs.com/2015/11/22/using-apache-sparkr-to-power-shiny-applications-part-i/

Join my FREE newsletter to see how other R packages could help in automated trading 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Flink vs Spark just stream data with Redis

Flink vs Spark just stream data with Redis

Honestly, why do they have these classic debates? Why not use Redis?  It is light, fast, bind to many languages, etc. Is it worth even reading this?

http://www.kdnuggets.com/2015/11/fast-big-data-apache-flink-spark-streaming.html

Join my FREE newsletter to learn how I use Redis for big data 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Skip the PhD learn spark for data scientist survey

Skip the PhD learn spark for data scientist survey

Once IBM got behind it, this framework has taken off

if you spend half of your time (4+ hours) doing basic exploratory analysis, you can expect your salary to drop by $6,609.

Here’s one big surprise: The language R didn’t fare particularly well, which was definitely not expected. The percentage of survey respondents who report using R fell from 57 in the 2014 survey to 52 this year. Spark, by comparison, grew 17 percent, while Scala grew 10 percent. (For what it’s worth, Hadoop’s use dropped from 19 percent to 13 percent, while Java dropped from 32 percent to 23 percent). …

While R remains one of the four primary tools used in data science (along with SQL, Excel, and Python), the usage pattern around R is changing, O’Reilly says.

“R is a prime example of a tool that is bridging the divide between open source and proprietary tools,” the report says. We’re seeing more interest in R paid by big software companies, including Microsoft, which bought Revolution Analytics, and Teradata, which just added support for R in its eponymous data warehousing environment.

This is reflective of changing times for R. “[T]he open-source-only crowd might be finding they don’t need such a large selection of tools, that Spark and Python do the job just fine,” O’Reilly says. “The large number of R packages has often been cited as a key advantage of R over tools such as Python, but this is not the kind of advantage that is guaranteed to last: there is no reason why developers of other open source tools can’t gradually build on their own libraries to catch up.”

You can download your copy of the O’Reilly 2015 Data Science Salary Survey here.

Skip the Ph.D and Learn Spark, Data Science Salary Survey Says

Join my FREE newsletter to see if Python help in my automated trading 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Does This Spark an Idea? Principal Component Analysis (PCA) of Yesterday and Today

Does This Spark an Idea? Principal Component Analysis (PCA) of Yesterday and Today

http://www.geniq.net/res/Principal-Component-Analysis-Yesterday-and-Today.html

Principal Component Analysis of Yesterday and Today geniq.net

 

=

we’ve just covered PCA in my Data Mining class, and so I’d be interested in reading more of your article. Thanks!

 

==
The article is actually a chapter in my new book
http://www.geniq.net/statistical-machine-data-mining-Bruce-Ratner-book.html .
However, if you rather not buy, I will be glad to chat with you.

==

 

Interesting. I am definitely interested in highly correlated eigengenes as a way to find expression networks.

 

==

PCA has long been a key part of my toolbox for dealing with high-dimensional data. At the start, it’s a good way to check whether samples cluster by treatment in a PCA biplot. Then I look at which variables have the highest loading on each principal component, which often gives an initial idea what the main biological trends in the response of the samples might be. At later stages I may filter variables by ANOVA then do PCA just using a subset of variables.

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!