Tag Archives: ticks

The proper way to convert market data ticks to OHLC

The proper way to convert market data ticks to OHLC

After my email of:

Convert tick data to OHLC?

I got this response from a long standing newsletter subscriber:

Super busy but my quick answer. A “bar” is just a collection of ticks of some kind. To build a “Minute” bar, you would collect ticks as they come in and store their values in the data structure of your choice. While building that collection you have you have to mark the O and C price and the H and L can change on tick as the structure fills. That data structure is usually an array of DateTimeOHLCV objects of some kind.

HOWEVER, most commercial trading software or developer API’s allow you to import data and have bar structures built out already(as you know). If the person asking the question needs to program bars, they plan on building a trading system scratch and if you can do that you can build a List<> object with values, I would imagine as the tick data structure is the backbone..literally. It sounds like this person should be using trading software like NinjaTrader or TradeStation, ect.. – those are very flexable/extendable and can import data. But there is an API with common trading functionality to write higher level code against as well.
Hope all is awesome!
Another response:
There is a plethora of similar questions and answers at. Stackover Flow  That’s where I’d start.

http://stackoverflow.com/questions/17952082/converting-tick-data-to-ohlc-bars-using-r

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

I don’t think IQFeed loses ticks

I dont think IQFeed  loses ticks

A comment cam in via Youtube

are you planning to continue using iqfeed as you move to linux does having to use wine worry you at all? and for using ib data (please note i am a complete beginner) i heard they sample their data, i.e. you dont get all ticks i am not sure if this is important in general, or just for tick by tick strategies so arent you concerned by this?

I do plan to use IQFeed with Linux using WINE. I know Interactive Broker aggregate data but it seems the IQFeed datas looks ok. This data seems to be streamed where there is no loss so I am not concerned. See the IQFeed presentation below. Either way with competitors, IQFeed is best value if you want Bloomberg Terminal level analysis.

 

Join my FREE newsletter to learn more about using IQFeed 

 

Demo:

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Download stock market ticks from Yahoo Finance

Download stock market ticks from Yahoo Finance

This is an Excel file exported out of my Postgres database. It is the first full 1.5 days of data capture using my this new program I modified. 

ticks

Join my FREE newsletter to see what else I come up with for this automated trading system 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

What is the best open source trading platform and database to hold ticks from cryptocurrency exchanges like Bitcoin?

What is the best open source trading platform and database to hold ticks from cryptocurrency exchanges like Bitcoin?

I thought this dialogue would be useful for those out there:

Really? I thought there was some “core” TradeLink that was to remain open source. I will check some more.

 

For starters, I just want to do what I did with TradeLink & IB already where I stored the tick data in a database, but with cryptocurrency. So I will have to write code to handle the cryptocurrency’s exchange API. I want to create my own tick database from the price information that I get from the exchange, and I also want to be able to chart the data. I thought you had said this was easy to do with MATLAB?

 

 

–> Tradelink does not use a database a special data format in flat files called TIK. You would probably need a connection to a decent database like MYSQL. Sorry for any confusion, you could leverage the power of Matlab for charting.Like https://www.youtube.com/watch?v=FRJ4zG4FksU

 

PS. In TradeLink’s terminology, what I want to do is record tick data from various cryptocurrency exchanges.

–> As said, the code for Tradelink has been obfuscated or something like that. All you need is a C# script to record the ticks of the Cryptocurrency ticks and record into a database. Correct? That is pretty straight forward but is there any sample code for the API? Should be if it was professional enough

 

I would not use Tradelink as the source has been closed off now. I would

suggest marketcetera as open source trading platform. It is in Java.

With Matlab, what are you specifically looking for? To connect to a

database?

 

PPS. I am starting to see some stuff about Tradelink being intentionally “crippled” by the original author, but apparently there is a fork:

–> Ok. That would be helpful so move ahead with that.

* Tradelink: Very capable software if you are looking for automated trading solutions. However, you should be aware of a few things. The original open source version is being taken in an unfavorable direction by the original author. There is a forked version that is worth seeking out. It is working from the last uncrippled version and is not restricted. You are probably on your own if you go with this software. Yet, at the same time it may be the best bet depending on your needs. Very capable platform to build off of.

–> aaah open source at its best

* Algo-trader: Seems very capable. I would probably go this direction if I was not tied to C++ and C#.

–> if this is the same AlgoTrader, this should be in Java. Is this same one you are reffering to ?

https://code.google.com/p/algo-trader/source/browse/#svn%2Ftrunk%2FAlgoTrader%2Fcode%2Fsrc%2Fmain

If you want go with something like Marketcetera

If you want C++, check out http://www.trading-shim.org/

Let me know what you go with!

 

> I have no idea about AlgoTrader, I just copied that from the same post
> that discussed Tradelink.
>
> What I read about Marketcetera did not sound that good. People seem to
> be saying that development is much less active than Tradelink. I would
> like to avoid Java, unless the Java platform is clearly superior.
>
> Isn’t trading shim what you had me try back in 2012? It was painful
> even to get to the first step 🙂
>
> So, I want to make sure I understand: To your knowledge Tradelink
> cannot record the tick data to a database, because it uses a flat file
> format? Maybe that is fine?
>
> I will play around for now, before committing myself to one solution.
>

 

Welcome to the world of open source trading. Sucks huh ? This is why I finally broke down to develop my own components. Anyhow, yes Tradelink uses flat files so the choice is limiting.

Join my FREE newsletter if we find ways to automate trading with Bitcoin 

 

 

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

See how fast Microsoft SQL Server with StreamInsight to retrieve over 1.2 million market data ticks

Hi there
See how fast SQL Server with StreamInsight to retrieve over 1.2 million market data ticks


https://quantlabs.net/blog/2013/09/see-how-fast-sql-server-with-streaminsight-to-retrieve-over-1-2-million-market-data-ticks/

At first when I saw this, I was very excited to see the results. After talking to the London Quant, it appears that further tests are needed to get see if this is the correct architecture to go with. I will release the results in coming days as I piece all this together. Stay tuned as this could be a good week.
Just a friendly message,
Bryan
PS. Remember my QuantLabs.net Premium Membership days for the public are now numbered!


–> JOIN NOW WHILE YOU CAN <–


Further info here.

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

See how fast SQL Server with StreamInsight to retrieve over 1.2 million market data ticks

See how fast SQL Server with StreamInsight to retrieve over 1.2 million market data ticks

This was pretty impressive to see!

Other helpful links:

http://blogs.msdn.com/b/streaminsight/archive/2012/07/09/streaminsight-2-1-meet-linq.aspx
http://code.msdn.microsoft.com/windowsdesktop/Using-SQL-Server-for-4b4f9c79

http://msdn.microsoft.com/en-us/magazine/hh205648.aspx

 

Getting Started With Microsoft StreamInsight

http://blogs.msdn.com/b/masimms/archive/2010/08/10/building-your-first-end-to-end-streaminsight-application.aspx

Follow me as I start to implement this SQL Server into my automated trading system

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Q&A on storing real time data ticks from IQFeed into a SQL Server while generating quant trading signals from DotNet and Matlab

Q&A on storing real time data ticks from IQFeed into a database while generating quant trading signals from DotNet and Matlab

This came in from a QuantLabs.net Premium Member:

>
> On top of that I have a quick question. I have both IQ feed and
> Matlab, do you know how to get market depth data work it to get
> trading signals? I’m also finding trouble to store the realtime data
> in a database.
>

 

My response:

Also, IQFeed only give Level 2 but you can get up to level 9 but that is big bucks (i.e. >$1000 per month. As for storage, you could try QCollector but I am working on a CEP solution to store with SQL Server.As for trading signals, you need to code that up into your trading model or strategy which I am also currently working on.
On 2013-09-11 16:21, Túlio Carnelossi wrote:

 

Interested in how I am handling all this? Join my FREE newsletter to learn more.

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Youtube video on Building a HFT platform in C with Redis In Memory NO SQL database for market data ticks

Youtube video on Building a HFT platform in C with Redis In Memory NO SQL database for market data ticks

Wow! This was easy. Join my FREE frequent newsletter to see what I do with this platform. 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

A complete Q&A from a potential quant member about Detlix, back testing, ticks, strategies, and trading platform

A complete Q&A from a potential quant member about Detlix, back testing, ticks, strategies, and trading platform

A potential member and visitor asked the following:

(Answers followed by my ->)

Writing this email to check some assumptions and ask some questions.

 

Am a rather new trader with math and 15 years of complex software

development.

Looking to become a profitable trader both discretionary and automated.

 

Very interested to use models you talk about on the site.

Looking to put some money to work.

Not afraid of hard work but must be profitable and not do it to satisfy

my software development and self education thirst.

 

–> Sure we want all that but no one can ever guarantee profitability

 

Would you say one can be profitable with the strategies and models your

group uses?

Are there such existing strategies that bring profits now?

 

–> I am about to start this analysis but this will take weeks (or months) but again, no one can ever guarantee profits. As profits prove themselves, it will add value to my membership which will then result higher monthly rates

 

Would you say they are production ready and able to trade real money?

If not then what does it take to take them there?

 

–> everything is production ready but again you will need to optimize the strategy code to your liking and trading goals. What I show is a foundation for now. My private member only webinars will show how to do this

 

Can you please tell me more about your business goals so I can better

gauge how I’d fit and help?

 

–> Like everybody profits but I am going slow and steady to understand each step of the way in great detail

 

More questions below if you care to answer them.

 

Thanks

Nick

 

1. Have you used NinjaTrader? I have some indicators useful for my

discretionary trading and would like to

reverse engineer & port them to a platform with better back testing

support.

Can you do this (reverse engineering) or others in your group?

–> I don’t use other platforms after long time learning, I would recommend working with your chosen broker’s API instead as that will save time in the long run

 

Not very sophisticated: fib lines, ATR based trend indicators, etc.

 

2. Do you use advanced strategy development tools like alphacet and

deltix?

 

–> I have been offered these but I am not interested in anything blackbox

I am looking at Deltix and it is pretty awesome.

Would you care to consider it for the group? (we’d share the costs)

–> I am not sure if this can be done in terms of having 70+ members sharing one license

 

Costs a lot per month due to expensive tick based data feeds but feels

very powerful.

 

3. What kind of strategies and models do you use?

 

– trend following

– counter trend

– breakout

– order-flow based

– tick data based

– volume based

– sentiment based (like ravenpack)

 

–> most will be quant based but the focus will be in order of priority of the results of this

http://quantlabs.net/surveys/2012/06/29/what-financial-forecasting-model-type-do-you-focus-on-within-r-or-other-statistical-tool/

 

If not then you interested in the above?

 

4. What data feed you use?

Tick or not?

 

–> both put prefer tick for live analysis but back testing needs bar

 

5. What backtesting tools, data feed and practices do you use?

–> I will be using parts of an open source HFT platform which has this in an automated way

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

intra day algo tracking price move ticks post news volatility,what are the best technical/time series indicators for pivot point detection

New to group,I have an intra day algo tracking price move ticks post news volatility,what are the best technical/time series indicators for pivot point detection (e.g. upswing after over sold)? thanks

 

==

I dunno if it is the very best, but amoung others, the one provided by tradestation is quite useful 😉
We are talking about swing pivots ( lower/higher lows/highs), not about Floor trader pivots, right ?

 

==

Hi many thanks yes swing pivots is what I am after, am current using 3-min interval moving average,but also wondered about post correlation to initial rise/fall, time periods?

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!