Category Archives: Quant Development

Quant Development

Tips to lower latency with cryptocurrency trading server on remote cloud traing server

Questions about building server for cryptocurrency and algo trading communnity

Here are some questions regarding my services and courses: (here are some tips on where to setup a remote cloud traing server)

See my Q&A video here

How to lower latency with cryptocurrency trading server with Kraken exchange

Here are some questions regarding my services and courses: (here are some tips on where to setup a remote cloud traing server)

See the detailed stats and video here

Thanks Bryan

P.S. I did not think I will be announcing this so early but here we are:

Whatapp chat group and Facebook so move to private chat server

Just a forewarning, i am no longer using Facebook or anything including Whatsapp or Instagrm or Messenger as of today.I will not replying on anything on any of those. If you want to join my private community, join here for a life access with a private chat server with no track or logging.

dvd
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Questions about building server for cryptocurrency and algo trading communnity

Here are some questions regarding my services and courses: (here are some tips on where to setup a remote cloud traing server)

Here is the links (for now):

Access the DVD and Private Chat server

dvd

Here is the Python infrastructure course talked about (this link will also change in the in near term)

Python Algo Trading Infrastructure with Crypto Currency

Here are some questions regarding my services and courses: (here are some tips on where to setup a remote cloud traing server)

Here is the links (for now):

Access the DVD and Private Chat server

dvd

Here is the Python infrastructure course talked about (this link will also change in the in near term)

Python Algo Trading Infrastructure with Crypto Currency

Get some free tech trading book https://quantlabs.net/

Talk to me about your trading https://quantlabs.net/questions/

Get lifetime access to our private chart server? https://quantlabs.net/dvd/

Hi Bryan,

Ok, I have done enough solo work… I’m ready to jump in the Quant Lab team!

Last fall summer didn’t quite exactly answer the question that I have.

The Python scripts that you do show in the video,

ccxt_market_data2.py

reportTA.py

I am interested in learning in the indicators that you do use:

The Fibonacci levels

Bollinger bands

The green Trend line that you draw in the *Trend.png

So do you confirm that in the “DVD” product, you will teach those topics (and perhaps also maybe share those 2 scripts)?

Beside the purchase of the DVD, what else do I need to purchase to be granted access to the signal service?

And why 2 distinct sets of signals? Binance ones arent good on Kraken and vice-versa??

BTW, after having viewed your video yesterday, I did create a ArchLinux VM with Vultr in the LAX region.

It did reduce my latency with Kraken from 140ms RTT with my Montreal server to something between 20-30ms RTT!

Lately, I was getting front-runned systematically on most of my triangular arbitrage trades, that was not fun…

I feel like now it is going much more profitable…

My setup is up and running but it hasn’t been triggered yet…(This strategy gets triggered in burst. It can be very quiet and start trading like crazy for short periods)

I’m very eager to see it in action! I did refine my system since last email exchange… I was struggling to reach $xx monthly trade volume. I got that milestone in the first few days of 2021 and I am about break the $xxx level… However, there is more volume than profit… (Not good)…

I need help from a team of liked minded individuals… I’m ready to join your team

Bottomline, I benefitted big time of that video. Thanks a lot! (I didn’t know about Vultr and they are much better than AWS…)

And, this is to be considered… I have way too much power on the VM. I took the 3 CPU High frequency setup. This thing is a beast. In fact, you don’t have 3 CPUs, you get 6 with the Hypethread thing… Despite processing millions of updates, I do use less than 10% CPU out of 600%!

If you have a hard time finding the right environment… Setting you an account on my VM could maybe be an option…

Hi Bryan,

Ok, I have done enough solo work… I’m ready to jump in the Quant Lab team!

Last fall summer didn’t quite exactly answer the question that I have.

The Python scripts that you do show in the video,

ccxt_market_data2.py

reportTA.py

I am interested in learning in the indicators that you do use:

The Fibonacci levels

Bollinger bands

The green Trend line that you draw in the *Trend.png

So do you confirm that in the “DVD” product, you will teach those topics (and perhaps also maybe share those 2 scripts)?

Beside the purchase of the DVD, what else do I need to purchase to be granted access to the signal service?

And why 2 distinct sets of signals? Binance ones arent good on Kraken and vice-versa??

BTW, after having viewed your video yesterday, I did create a ArchLinux VM with Vultr in the LAX region.

It did reduce my latency with Kraken from 140ms RTT with my Montreal server to something between 20-30ms RTT!

Lately, I was getting front-runned systematically on most of my triangular arbitrage trades, that was not fun…

I feel like now it is going much more profitable…

My setup is up and running but it hasn’t been triggered yet…(This strategy gets triggered in burst. It can be very quiet and start trading like crazy for short periods)

I’m very eager to see it in action! I did refine my system since last email exchange… I was struggling to reach $xx monthly trade volume. I got that milestone in the first few days of 2021 and I am about break the $xxx level… However, there is more volume than profit… (Not good)…

I need help from a team of liked minded individuals… I’m ready to join your team

Bottomline, I benefitted big time of that video. Thanks a lot! (I didn’t know about Vultr and they are much better than AWS…)

And, this is to be considered… I have way too much power on the VM. I took the 3 CPU High frequency setup. This thing is a beast. In fact, you don’t have 3 CPUs, you get 6 with the Hypethread thing… Despite processing millions of updates, I do use less than 10% CPU out of 600%!

If you have a hard time finding the right environment… Setting you an account on my VM could maybe be an option…

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to lower latency with cryptocurrency trading server with Kraken exchange

Here are some questions regarding my services and courses: (here are some tips on where to setup a remote cloud traing server)

Here is the links (for now):

Access the DVD and Private Chat server

dvd

Here is the Python infrastructure course talked about (this link will also change in the in near term)

Here is the set of queries from someone:

Get some free tech trading book https://quantlabs.net/

Talk to me about your trading https://quantlabs.net/questions/

Get lifetime access to our private chart server? https://quantlabs.net/dvd/

Here is the corespondent

Hi Bryan,

Ok, I have done enough solo work… I’m ready to jump in the Quant Lab team!

Ok, so based on what you wrote,

without having the exact implementation details that you do use, your python algo trading class is covering :

Fibonacci, Bollingers and how to calculate a trend line like the one your script reportTA generates (and much more, I assume)

and the course would give good knowledge on how to do it myself. Correct?

2. I did ask Kraken reps themselves about what someone could do to have good latency with their service and they told me the following:

Low Latency Options


For our exchange REST and WebSocket APIs, we recommend hosting your API client at AWS in California.

Of course, as you know, AWS sucks. They offer only old platforms like CentOS 6 (based on 2.6.32 kernel!)

The problem is that Kraken is behind Cloudflare. When you do a DNS lookup on their servers, you get a Cloudflare IP. Their network topology is totally opaque to regular net tools. You either have insider knowledge from Cloudflare, or you figure it out with trial and errors. Let me show you something.

ping numbers from MTL:

$ ping ws-auth.kraken.com

PING ws-auth.kraken.com (104.16.212.191) 56(84) bytes of data.

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=1 ttl=60 time=0.652 ms

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=2 ttl=60 time=12.8 ms

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=3 ttl=60 time=0.430 ms

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=4 ttl=60 time=0.449 ms

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=5 ttl=60 time=0.488 ms

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=6 ttl=60 time=0.461 ms

64 bytes from 104.16.212.191 (104.16.212.191): icmp_seq=7 ttl=60 time=0.484 ms

^C

— ws-auth.kraken.com ping statistics —

7 packets transmitted, 7 received, 0% packet loss, time 6010ms

rtt min/avg/max/mdev = 0.430/2.258/12.844/4.322 ms

ping from vultr LAX site:

$ ping ws-auth.kraken.com

PING ws-auth.kraken.com (104.16.215.191) 56(84) bytes of data.

64 bytes from 104.16.215.191 (104.16.215.191): icmp_seq=1 ttl=58 time=0.783 ms

64 bytes from 104.16.215.191 (104.16.215.191): icmp_seq=2 ttl=58 time=0.860 ms

64 bytes from 104.16.215.191 (104.16.215.191): icmp_seq=3 ttl=58 time=0.794 ms

64 bytes from 104.16.215.191 (104.16.215.191): icmp_seq=4 ttl=58 time=0.412 ms

^C

— ws-auth.kraken.com ping statistics —

4 packets transmitted, 4 received, 0% packet loss, time 3004ms

rtt min/avg/max/mdev = 0.412/0.712/0.860/0.175 ms

That looks like pretty solid #s right, they fool you. They represent the ping time you get from the nearest Cloudflare proxy.

I use Kraken WS updates ts and I measure the difference between a book entry introduction and the order open timestamp as reported by the exchange ack msg (assuming my bot reaction time is negligeable. it is by being less than 1ms).

From Montreal, I was having interval of 140ms

So far since this morning, the best RTT that I did measure from LAX was 60ms.

It is much better but I’m not yet 100% satisfied as, again with the help of Kraken updates timestamps, I know that some traders are able to get roughly a 35-40ms RTT…

If there is more than 1 trader doing triangular arbitrage, the fastest one reap it all…

Yes, it is very tough… The concept is simple but there is a lot of problems in the details…

I have been on it for the last year and it starts to work… I have worked out most of the pitfalls.

3. I’ll take a look at Lnode but TBH, I don’t really care about customer service… I mostly care about the product and I have been impressed a lot by vultr. In less than in an hour, I was up and running with a fresh server install with exactly the same environment than my dev env. My previous server in mtl was CentOS 7 (My webserver) and it was pain to rebuild my bot for that different env. With vultr machine, I just need to upload the binary that I have compiled on my desktop and it works out of the box with the added security benefit of not having to move my source code containing my secret sauce and my exchange accoutn keys on a different machine.

I need to find the data center that is going to provide me a latency that is less than 10ms… If I can find that, I’ll do my own customer service 😉

Hey, I have found this blog:

https://www.sissoftwarefactory.com/blog/

very interesting stuff…

To wrap things up:

1. Is your python algo trading course is going to meet my needs of learning the indicators that you show in your videos?

2. What is the product granting access to your signal service?

Last fall summer didn’t quite exactly answer the question that I have.

The Python scripts that you do show in the video,

ccxt_market_data2.py

reportTA.py

I am interested in learning in the indicators that you do use:

The Fibonacci levels

Bollinger bands

The green Trend line that you draw in the *Trend.png

So do you confirm that in the “DVD” product, you will teach those topics (and perhaps also maybe share those 2 scripts)?

Beside the purchase of the DVD, what else do I need to purchase to be granted access to the signal service?

And why 2 distinct sets of signals? Binance ones arent good on Kraken and vice-versa??

BTW, after having viewed your video yesterday, I did create a ArchLinux VM with Vultr in the LAX region.

It did reduce my latency with Kraken from 140ms RTT with my Montreal server to something between 20-30ms RTT!

Lately, I was getting front-runned systematically on most of my triangular arbitrage trades, that was not fun…

I feel like now it is going much more profitable…

My setup is up and running but it hasn’t been triggered yet…(This strategy gets triggered in burst. It can be very quiet and start trading like crazy for short periods)

I’m very eager to see it in action! I did refine my system since last email exchange… I was struggling to reach $xx monthly trade volume. I got that milestone in the first few days of 2021 and I am about break the $xxx level… However, there is more volume than profit… (Not good)…

I need help from a team of liked minded individuals… I’m ready to join your team

Bottomline, I benefitted big time of that video. Thanks a lot! (I didn’t know about Vultr and they are much better than AWS…)

And, this is to be considered… I have way too much power on the VM. I took the 3 CPU High frequency setup. This thing is a beast. In fact, you don’t have 3 CPUs, you get 6 with the Hypethread thing… Despite processing millions of updates, I do use less than 10% CPU out of 600%!

If you have a hard time finding the right environment… Setting you an account on my VM could maybe be an option…

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Trading client works with production ready Python web server with Binance

New trading client works with production ready Python web server with Binance bot bank

Based on my last Youtube video, some alt coins have returned 4-12x return regardless of the recent Bitcoin crash. As a result, there are lots opportunity out there. The big news is that this message queue is back to working with a production ready Python web server. I have also tried out a better grade cloud provider I promoted on my blog in the last few days.

How is Bitcoin Ethereum Alt Coins trading been doing?

has been a while ot looking at these trading analytics. I also cover the most profitable coins you never heard of on Binance

See the video here

Thanks Bryan

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Trying Vultr with instructions to set up Ubuntu Linux server with Python

I have decided to go with Vultr Cloud Compute virtual machine server as at vultr.com. It is only $20 per month! Digital Ocean has customer service problems with their virtual machines are slow and can be corrupted easily.

It is quite affordable but here are some instructions to set up your virtual machine. I also chose Ubuntu 20.1 for the Linux distribution.

Add user https://linuxize.com/post/how-to-create-users-in-linux-using-the-useradd-command/

Set user password
https://www.cyberciti.biz/faq/linux-set-change-password-how-to/

You should be able to ssh in as any other server

sudo apt-get update

To install an exact version of Python
https://unix.stackexchange.com/questions/332641/how-to-install-python-3-6

Ubuntu server

Add user to sudoers
https://www.digitalocean.com/community/tutorials/how-to-create-a-sudo-user-on-ubuntu-quickstart

Install Pip
https://linuxize.com/post/how-to-install-pip-on-ubuntu-18.04/

Use SCP to transfer big Zipped files

https://www.vultr.com/docs/securely-transfer-files-over-the-private-network-using-scp-or-rsync

Install nginx
https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-open-source/

Get some free tech trading book https://quantlabs.net/

Talk to me about your trading https://quantlabs.net/questions/

Get lifetime access to our private chart server? https://quantlabs.net/dvd/

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Digital Ocean RDP VM vs cloud vs Docker containter

There are basically 3 server deployment options as explained below. The Linux set option is horrendous. Cloud is better but still complicated. Screw it, I will be lazy to just set up a remote desktop environment via an Ubuntu Linux server on Digital Ocean.

Setting up Flask app on Heroku cloud. My traditional don’t have the necessary software (e.g. Gunicorn) nor will support it. It is time to move this.

https://devcenter.heroku.com/articles/python-gunicorn

Using packages and Pip (Cheeseshop?)

https://devcenter.heroku.com/articles/python-pip

HIgh level options to deploy  https://blog.miguelgrinberg.com/post/how-to-deploy-a-react–flask-project

As you can see in this mega tutorial chapter 17-19, highlight the different deployment options of Linux, Heroku, and Docker. I think Heroku is most logical and easiest with minimal amount of work. Linux is tough like in my case espescailly if your hosting company offers limited support. It seems Docker container hosting can be very expensive but easy to deploy from server to server.

https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvii-deployment-on-linux

I like the Docker options since the app is in a self contained container which can be transported from server to server quite easily. Here are some hosting options:

I just discovered this: Create a remote desktop virtual machine as with Ubuntu Desktop Linux. This is possible with Digital Ocean as with here:

https://www.digitalocean.com/community/questions/how-can-i-create-a-vm-with-ubuntu-desktop-in-digital-ocean

Digital Ocean take Paypal as I like as well.

Heroku got quite a few set up of upvotes when mentioned which is encouraging. I do like the idea of turning a ‘cheap’ virtual machine into a Docker container to have it hosted somewhere.  I wonder if

Heroku also take Paypal out of convenience.

Heroku vs AWS (just go to the verdict at the bottom)

Bonus:

You always hear about this serverless. Here is a good article with Amazon’s AWS

https://blog.miguelgrinberg.com/post/serverless-deployments-of-python-apis

This guys articles are definitely top notch when it comes to Flask technology.

(as I hinted before, you would want to stay away from this proprietary tech so you are no held hostage to AWS. Can your say Parler anyone?) It is an interesting read though.

From a career perspective, knowing these Python technologies can help your career since it is the most popular for web development.

https://insights.dice.com/2020/07/30/top-24-programming-languages-web-development-python-more/
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

The struggle is real with any Flask production server

Lots of late nights with the code to deployed on any Flask production server

I looked at pretty well at every Flask server for my server on my hosting company which uses CentOS 7. The only one I got working was this server was uWSGI. This was using this

The struggle is real.

Thanks

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

The only Flask production server I got working was uWSGI

I looked at pretty well at every Flask server for my server on my hosting company which uses CentOS 7. The only one I got working was this server was uWSGI. This was using this

https://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.html

This was in conjunction with this

It seems to be a complete mess. I just need to ensure I have enough threads to cover all my suer of this Analytics for each user.

There might be a way to avoid presetting a number of threads when launching. This is explained with the –lazy flag here:

https://stackoverflow.com/questions/51931262/threading-in-flask-not-working-with-uwsgi-but-working-on-commandline

It seems uWSGI is as fast Gunicorn according to this:

https://blog.miguelgrinberg.com/post/ignore-all-web-performance-benchmarks-including-this-one

It seems you can easily configure the number of processes needed quite easily.

https://stackoverflow.com/questions/40235373/serving-a-wsgi-app-endpoint-in-a-separate-thread

Get some free tech trading book https://quantlabs.net/

Talk to me about your trading https://quantlabs.net/questions/

Get lifetime access to our private chart server? https://quantlabs.net/dvd/

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Python Flask production server options Gunicorn vs Waitress

Here is a common error I get with Flash Waitress web server I am currently testing: (should I go with Gunicon?)

ERROR:waitress:Socket error
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/waitress/channel.py”, line 118, in handle_write
flush()
File “/usr/local/lib/python3.6/site-packages/waitress/channel.py”, line 207, in _flush_some_if_lockable
self._flush_some()
File “/usr/local/lib/python3.6/site-packages/waitress/channel.py”, line 227, in _flush_some
num_sent = self.send(chunk)
File “/usr/local/lib/python3.6/site-packages/waitress/wasyncore.py”, line 433, in send
result = self.socket.send(data)
TimeoutError: [Errno 110] Connection timed out

I tried to be compliant with LiteSpeed which my hosting provider will support. There are a few ways to do this with:

https://www.litespeedtech.com/docs/lsapi/python

This commerical Litespeed link should confirm this OpenLiteSpeed should work

https://www.litespeedtech.com/support/forum/threads/i-am-not-able-to-run-python.18528/

This option with LIteSpeed was not possible. I did however, get uswgi server working from this example on my web hosting provider:

https://riptutorial.com/flask/example/16286/using-uwsgi-to-run-a-flask-application

Here are some links to consider

Host it on Heroku? https://www.heroku.com/pricing
Good general overview:
https://vsupalov.com/flask-web-server-in-production/

I am using Waitress but it seems to time out quite a bit. This above article recommend \s Gunicorn.

https://www.toptal.com/flask/flask-production-recipes

Second article to recommend both Gunicorn but use Nginx instead?

How configure Nginx for Flask
https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uswgi-and-nginx-on-ubuntu-18-04
Another Gunicorn with Nginx
https://medium.com/faun/deploy-flask-app-with-nginx-using-gunicorn-7fda4f50066a

A warning about databases for Gunicorn
https://github.com/etianen/django-herokuapp/issues/9

Comparison across other Python web servers
https://www.appdynamics.com/blog/engineering/a-performance-analysis-of-python-wsgi-servers-part-2/

https://www.reddit.com/r/Python/comments/76g3vf/which_wsgi_servers_are_safe_to_expose_to_the/
This says Waitress is safe
https://docs.pylonsproject.org/projects/waitress/en/latest/design.html

Pros for Waitress over Gunicorn
https://www.reddit.com/r/Python/comments/qupyf/waitress_productionquality_purepython_wsgi_server/

Gunicorn vs Waitress https://stackshare.io/stackups/gunicorn-vs-waitress

Overall list of Python production ready servers

https://towardsdatascience.com/my-favorite-python-servers-to-deploy-into-production-d92289764fbe

Here is an overall overview with example scripts included:

http://fgimian.github.io/blog/2012/12/08/setting-up-a-rock-solid-python-development-web-server/

Gunicon is top and Waitress is second. Let me give Gunicorn a shot in a way to eliminate these socket errors.

This appears to be a simple way to setup Gunicon with Nginx

View at Medium.com

For CentOS, I had to install Development Tools

Install Build-Essentials for CentOS, RHEL and Ubuntu

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

No more proprietary 3rd party software or big tech cloud

After the recent online purge of I will not supporting these big tech companies any more. There is a rant alert for this!

Get some free tech trading book https://quantlabs.net/

Talk to me about your trading https://quantlabs.net/questions/

Get lifetime access to our private chart server? https://quantlabs.net/dvd/

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!