How to Build a historical database with Google or Yahoo Financee
The following command line (or command prompt) instructions work mostly for Windows but theoretically could be used for Mac OSX or Linux.
1. Use Yahoo Stock Symbols (YSS) to generate file with 34,000 symbols or however needed.
You need Python language installed but use: python get_yahoo_stock_symbols.py -v -c 5
This is for 5 columns so it will do the full 34,000 symbols. Read the readme file for instructions.
Should generate file name like yahoo_stock_symbols_5
2. Download wget for Linux or mac or wget for Windows. From:
Use something like
where ABB is symbol and generated needed file but could have a program loop through generated YSS file yahoo_stock_symbols_5.
3. Install SQL Server or SQL Server Express. For each symbol generated
CSV file, use bcp to import directly into SQL Server. More into at:
Note you could use any other database really but I like SQL Server.
You could also Cron job to initiate wget process for desired symbols.
Notes about wget downloading from Google Finance:
It is better to download from Yahoo as there seems to be NO download frequency limit.
Notes about Google and Yahoo Finance services
Posts the EOD data earlier than Yahoo (2-3 hours before Yahoo posts the data).
Sample URL: http://www.google.com/finance/historical?q=NYSE:GOOG&output=csv
Has a limit on how many requests you can make per minute (don’t know
the exact figure, but if you request data too often google will block
you for a period of time).
Good if you download the data manually and automatically (at low frequency).
Posts the EOD data a little later (around 7:00 p.m. CST).
Sample URL: http://ichart.finance.yahoo.com/table.csv?s=YHOO&d=3&e=23&f=2010&g=d&a=3&b=12&c=1996&ignore=.csv
Has no limit on how many requests you can make at all!
Good if you download the data both manually and automatically (no
Use http://www.gummy-stuff.org/Yahoo-data.htm from more details on Yahoo data.NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!