Tag Archives: stream

Q & A on what is the best way to stream Excel data into SQL database table with DotNet CSharp?

Q & A on what is the best way to stream Excel data into SQL database table with DotNet CSharp?Answers below:
> Hi Bryan,
>
> Your cswatcher is watching csv file, can you suggest direction for
> watching excel file?

–> I am using now  *** for  Excel which does streaming withj IQFeed or IB

Is it good to convert excel file to csv then
> import into sql?

–> You could easily export the Excel into SQL via C#

Can SSIS watch excel file and streaming changed data
> into new record in sql table? What is the best way?

–> I have never used SSIS but this might be a decent option. I currently wrote a pair trading C# application which build Excel worksheets using the legacy Interop API. it is very slow and not recommended. I have also seen this as well for Office 2013:
http://msdn.microsoft.com/en-us/library/bb448854(v=office.14).aspx
This might be the lightest and modern way to use C# to populate your SQL table.

Join my FREE newsletter to learn more about .NET

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to use IQFeed tick data stream into MYSQL for a Matlab Simulink trading model

How to use IQFeed tick data stream into MYSQL for a Matlab Simulink trading model

My original plan to use Simulink project which was an HTML/XMLRPC server appears to be either too old or I am pretty inept on how to get this working. I am not planning to use the Matlab Database Toolbox to have an active MYSQL connection into a table of tick data. This table will be fed basically by a queue of data being streamed in by a working example of IQFeeds API with Java. Kind of basic and primitive but should work for now. If I could afford something like kdb, I would love to use it as a way to house tick data. Anyhow, I will need to revisit as this potential framework moves toward a production environment. I could replace MYSQL with SQL Server for better performance but I think it would be farily marginal unless someone shows me how to better optimize this stack.  MYSQL is also free as you know.

I could use the database connection to export to Matlab’s workspace which the the Simulink model will listen to. This is just for testing again.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Useful documents for Matlab Stateflow event stream processing with FPGA uses in HFT for ultra lowest latency

Useful documents for Matlab Stateflow event stream processing with FPGA uses in HFT for ultra lowest latency

Interested in learning more? Join my FREE newsletter

From my Youtube user so thanks to him:

A bunch of documents relating to Stateflow for event/stream processing:

The main one:
http://www.patentgenius.com/patent/7966161.html
http://www.google.com/patents/US7966161

Others:
http://www.google.com/patents/EP1817700A2?cl=en&dq=%22stateflow%22+event&hl=en&sa=X&ei=pStpUdHGCcip0AG0xYCoDA&ved=0CCkQ6AEwAA

http://www.google.com/patents/US7500209?pg=PA10&dq=%22stateflow%22+event&hl=en&sa=X&ei=pStpUdHGCcip0AG0xYCoDA&ved=0CCwQ6AEwAQ

http://www.google.com/patents/US7925477?pg=PA21&dq=%22stateflow%22+event&hl=en&sa=X&ei=pStpUdHGCcip0AG0xYCoDA&ved=0CEEQ6AEwCA

http://www.google.com/patents/US8266584?dq=%22stateflow%22+event&hl=en&sa=X&ei=oCxpUdXODO-50QG4woDAAQ&ved=0CCkQ6AEwADgU

http://www.google.com/patents/US7433808?pg=PA27&dq=%22stateflow%22+event&hl=en&sa=X&ei=hyxpUfioHsav0AG4zYH4AQ&ved=0CDgQ6AEwBTgK

http://www.google.com/patents/US7640154?pg=PA21&dq=%22stateflow%22+event&hl=en&sa=X&ei=hyxpUfioHsav0AG4zYH4AQ&ved=0CDIQ6AEwAzgK

Stream:
http://www.google.com/patents/US20120291054?dq=%22stateflow%22+%22stream+processing%22&hl=en&sa=X&ei=FSxpUYT3LLOG0QGRz4HQBA&ved=0CCkQ6AEwAA

Info about objects:
http://www.google.com/patents/US8392151?dq=%22stateflow%22+event&hl=en&sa=X&ei=pStpUdHGCcip0AG0xYCoDA&ved=0CD4Q6AEwBw

This is for FPGA event processing:

http://people.inf.ethz.ch/lwoods/publications/Complex-Event-Detection-at-Wire-Speed-with-FPGAs.pdf

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to stream your historical market tick data for back testing your trading strategy? KDB or Microsoft SQL Server?

How to stream your historical market tick data for back testing your trading strategy? KDB or Microsoft SQL Server?

From a Youtube user:

Hey, i have a question. If i have lots of historical tick data, and want to replay the data as it happened, i.e. like a tick player, a peice of software that actually replays the ticks as they happened, how do i do this? Other than ninjatrader?

My answer:

Try kdb if you can afford it. Or this:

http://www.microsoft.com/en-us/sqlserver/solutions-technologies/business-intelligence/streaming-data.aspx

This is considered prettty well by the quants

Want to learn more? Join my FREE newsletter

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Use Metatrader 4 MQL4 to stream minute ticks for free with demo market account

Use Metatrader 4 MQL4 to stream minute ticks for free
Well this software is back in my good books. I have figured out a way to stream minute tick data (or any chart of your choosing). It ain’t hard but I don’t know why I never thought of it before. Yahoo has severely limited ways of streaming its data. You can only do this symbol by symbol individually but it is a smart way to stream forex pairs at least.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Job Interview question: How to stream output, open file, and read file in C++ and C

Job Interview question: How to stream output, open file, and read file in C++ and C

To stream to clog, set:
#include
#include

void main ( )
{
using namespace std;

// open the file “file_name.txt”
// for reading
ifstream in(“file_name.txt”);

// output the all file to stdout
if ( in )
cout << in.rdbuf(); else // if the ifstream object is in a bad state // output an error message to stderr clog << "Error while opening the file" << endl; } How to open and read a file: #include

void readone();

int main(int argn, char **argv)
{
int i;
for (i=0; i<=2; i++){ readone(); } } void readone() { static int ctr = 0; FILE *f = fopen("test.txt", "r"); char str[100]; int i; for (i=0; i<=ctr; i++){ fscanf(f, "%[^\n]\n", str); } puts(str); fclose(f); ctr++; } Beaware of ftell and fseek: /* ftell example : getting size of a file */ #include

int main ()
{
FILE * pFile;
long size;

pFile = fopen (“myfile.txt”,”rb”);
if (pFile==NULL) perror (“Error opening file”);
else
{
fseek (pFile, 0, SEEK_END);
size=ftell (pFile);
fclose (pFile);
printf (“Size of myfile.txt: %ld bytes.\n”,size);
}
return 0;
}

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

C++ interview question on stream like ftell, fseek, read file

C++ interview question on stream like ftell, fseek, read file

Stream:
you are writing a function that needs to be able to write to both standard output
(typically the console screen) and files. which one of the following function
declarations satisfies that needs?

a. void print (cout);
b. void print (istream &is);
c. void print (istream is);
d. void print (ofstream ofs);
e. void print (ostream &os);

Its d since stdout is global we don’t need to pass that only the reference to file is local hence it is needed by the function

e.
cout is an object of ostream
ofstream is a specilization of basic_ofstream which is derived from basic_ostream

e is wrong . d is correct. We have to pass the descriptor of the file only and not cout. As cout is a global object we can use it in the function without passing it as the parameter

what is standard narrow char of iostream object
If the question is meant to read:
What are the names of the standard narrow-character iostream objects?
the answer would be:
cin, cout, cerr, clog
but I’m not sure if that is what the question is asking or not.
char narrow ( char c, char dfault ) const;
This is a function in library ios.

A Narrow character converts localized parameter c to its standard char equivalent.
c: character to be narrowed.
dfault: default character to be returned if c has no char equivalent

Virtual Functions are implemented usign the vtable, which is a table of function pointers for each of the virtual function present in the class.
The class creates a vtable upon call of its constructor. In case of derived class ,base class creates the initial vtable but constructor of derived class over-rides the entry in the vtable if the virtual function is defined in derived class.

The C standard library specifies calls ftell() and fseek() that allow you to save and restore the position in the file. (Note: Microsoft probably has their own bastardized equivalents.) Just store the offset in a static variable.
static char *readone(char *path) {
static long offset;
static char line[8192];
FILE *f = fopen(path, “r”);
char *p;
fseek(f, offset, SEEK_SET);
p = fgets(line, sizeof line, f);
offset = ftell(f);
fclose(f);
return p;
}

int main(int argc, char **argv) {
char *line;
while((line = readone(argv[1])) != NULL) {
fputs(line, stdout);
}
}

It seems to a pretty neat approach. Does anyone sees anything missing here.
@hary. Everything is missing here.

It gives incorrect answer! It has buffer overflow. The first time readone is called, it could seek to a random offset in the file.

In short, it is crap.
If i wrong or missing something please correct me

I agree buffer overflow would be there. But since the question says readoe would read one line at every call, then i am assuming they there is a restriction on the size of each line, because you need to read each line and store in internal buffer. Any API one uses, fscanf, fgets will make use of a buffer string.
As far as the first call i sconcerned it will not seek to a random offset because offset is static and is initialized to 0, so fseek will move you nowhere w.r.t to the begining of the file.
If you still fell its wrong suggest ways to improve this.

it happens, its all abt keeping ur nerves. Q is simple and instead of algo, I guess intent was to see the design/architect skills.
I hv very little idea of C/C++ lib for readfile

In C# I would use static counter to keep the count of number of lines read
Then some additional checks like EOF case/ExceptionHandling/Empty lines etc.

Keep trying, everyones knows its hard to get thru, but once u get it, its an accomplishment.

Gud Luck
I am sorry, I missed to mention, the interviewer wanted me to use fscanf!
Here is the solution if you forget (or don’t know about) the fseek.

string readone(char* filename) {
static int ctr=1; // counter to track how many times readone is called.
string buf;
ifstream in(filename);
for(int i=0;i

void readone();

int main(int argn, char **argv)
{
int i;
for (i=0; i<=2; i++){ readone(); } } void readone() { static int ctr = 0; FILE *f = fopen("test.txt", "r"); char str[100]; int i; for (i=0; i<=ctr; i++){ fscanf(f, "%[^\n]\n", str); } puts(str); fclose(f); ctr++; } B on March 18, 2010 |Edit | Edit fscanf(f, "%[^\n]\n", str); should have been... fscanf(f, "%[^\n]s", str); //Read not-until '\n'(newline) char. doesnt fgets function read a line from a file.. i mean read is terminated as soon as it encounters a /n No I am so sorry to write an incorrect answer. Answer to your question is yes Here is the definition from man pages fgets() reads in at most one less than size characters from stream and stores them into the buffer pointed to by s. Reading stops after an EOF or a newline. If a newline is read, it is stored into the buffer. A '\0' is stored after the last character in the buffer. What i intended was that you anyhow would be restricted by SIZE chars and so cap on the buffer used must always be there unless we think of a way to avoid this. Guys, from my Msft interview experience last year, I can safely assure you that using fseek here would definitely land you with a reject. They tend to overlook candidates who use a lot of library functions while operating on strings. My 2 paisa! Guys, Sincerely, the solution to this question can be from very simple C++ to very complex C. I think it depend on the interviewer's preferences. Below is a simple C++ solution using fstream library. You don't need fseek here as fstream keep tracking it for you. In fact, it always does and we often overlook this as we often use Yan's approach all the times. fstream file; void readOne() { string str; getline(file,str); cout << str << endl; } int _tmain(int argc, _TCHAR* argv[]) { file.open("Your file path here",std::ios_base::in); readOne(); readOne(); readOne(); file.close(); string str; cin >> str;
}
Go through all the C++ Qs put on careercup , in online test most of these Qs repeat .

// Online Test
class Someclass {
public:
int x;
public :
Someclass(int xx) : x(xx) { }
Someclass(const Someclass& a) { x = a.x ; x++;}
Someclass& operator =(const Someclass& a1) { x = a1.x ; x–;}
};
int main( )
{

/*Someclass a(4);
Someclass b = a;
}
Q. What will be output
6
b.x = 4

Um… actually the answer is:
b.x = 5
because in this case, the copy constructor is called. In no case could the result be 4 since “=” symbol forces either the copy constructor or the assignment operator to be invoked. The statements
Someclass c(2);
c = a;
will give the result 3, not 4 because the assignment operator will be called rather than either of the constructors.

Mimi on March 01, 2010 |Edit | Edit

For the second case, I think you are talking about 1 instead of 3. Because “=” is called.

No… the assignment operator will be called to assign a to c. In this context the value of 2 contained in c.x will be overwritten by the value of 4 contained in a.x. This new value will then be decremented to 3. I suggest you compile and run the following example
#include

using namespace std;

// Online Test
class Someclass {
public:
int x;
public :
Someclass(int xx) : x(xx) { }
Someclass(const Someclass& a) { x = a.x ; x++;}
Someclass& operator =(const Someclass& a1) { x = a1.x ; x–;}
};

int main( )
{

Someclass a(4);
Someclass b = a;
Someclass c(2);

c = a;

cout << "a.x = " << a.x << endl; cout << "b.x = " << b.x << endl; cout << "c.x = " << c.x << endl; } chethanjs on May 07, 2010 |Edit | Edit "=" is such a b*tch! 🙂 I actually compiled your (dfmcia) program. You are right- b.x =5. I expected b.x=3. You said for "=" operator either the copy or assignment constructor is called, but i always got b.x=5 when I ran the program multiples. How do i make sure only assignment operator is called when "=" is used, given that both of them are defined? Forget it, I figured it out. 1) When you are using "=" when initializing a class, then always copy constructor is called. Eg: Someclass b = a; 2) When you are using "=" to assign to an already initialized class, then assignment operator is called. Eg: b = a or c = a. //Online Test extern void print(int *ia, int sz); void print(int *ia, int sz); Q. Will it compile Are the declarations in the same file? same file wont compile Yes ,both pick up internal linkage Actually it depends on whether you use the function or not. This will compile with both statements because the local declaration shadows the scope of the external declaration. However, if the function is actually used, there has to be an actual definition of the function somewhere. Neither of these two statements provide such a definition. trm on March 18, 2010 |Edit | Edit It compiles, but doesn't link if the funciton is called. When extern is put in front of a variable it means that the memory for the variable is some where else.When extern is put before a function, what exactly does this mean ? //Online Test struct A{ int i , j; A(int ii , int jj) :i(ii),j(ii){} A(const A&a){ cout << a.i << a.j; } void operator = (const A& a){ cout <
bool is_int();
is_int must return true if T is int and false otherwise. How do you
implement is_int?
Choice 1

template
bool is_int()
{
return T is int;
}
Choice 2

template
bool is_int()
{
return false;
};
template
bool is_int()
{
return true;
};
Choice 3

template
bool is_int()
{
T a;
return dynamic_cast(&a) != 0;
}
Choice 4

template
bool is_int()
{
return typeof(T) == typeof(int);
};
Choice 5

template
bool is_int()
{
return false;
};
template<>
bool is_int()
{
return true;
};
Choise 5
Template specialization
Choice 4.

Although, I forget if the keyword is “typeid” or “typeof”. I think “typeid”.

I believe choice 4 better than choice 5.

It is typeid – Thats why its 5) 😉
Yes, my brain seems to get smarter by the day, I guess (hope) and its another day.

Inner class:
ns an instance of the outer class. You could also NOT provide any setter methods in the outer class, so that the outer class can be constructed only through the inner class build method

Java final equivelant in C++
1. What does the ‘final’ key word do in Java?
2. Does C++ provide similar machanism? If not, how would you implement it?

final class in java : does not allow a class to be subclassed.

c++ , we can make use of private keyword and make all data in a class private ie. constructor , methods and data needs to be declared in private section. There is no keyword that we can use on class to prevent it from subclassing.

final keyword for variable : in c++ make it const , i guess.

final methods : In c++ , we cannot avoid subclasses overloading or overriding it.
let me know if i m wrong.

In c++, we make a same effect by declaring the class to private class. Even if we can inherit with private inheritance, there is no meaning because subclass cannot use or access any memeber in that class.
(2) final method: cannot overload it. In c++, we can declare the method as private. Similarily, the subclass cannot access the method.
(3) final variable: one time initialization constant variable. In c++, we can define a reference variable for the same effect as one time initialization and never updated its initial value.

I dont agree with drwolf on the

point 1), you should just make the constructor as private and that will prevent the derivation of the class.

and 3) we need to use the keyword “const” to make sure that the contents of the reference do not change, because const doesnt allow the programmer/application to change the value.

If you make the constructor private then how will create an instance of the class?
Anonymous on January 03, 2010 |Edit | Edit

#include

class foo
{
foo();
};

int main()
{
foo f;
return 0;
}

output:
test-final-class.cpp: In function ‘int main()’:
test-final-class.cpp:5: error: ‘foo::foo()’ is private
test-final-class.cpp:10: error: within this context

This is funny. I see that one of the replies got a -1! If I recall correctly, this question was posted around 5-6 times (currently it seems to appear two times). I would have expected positive votes for that response.

Declare that class in unnamed namespace.

i agree with desi and i must say what drwolf is also correct meaning thereby,
#. if yuo just make the constructor private then its enough for any deriving class to not make any subclass on that, and we don’t need to do other things as mentioned by drwolf, but that doesnot say his answer is wrong, drwolf add extra steps to do the same, which can be avoided.
# yeh, the reference has to be declared as const else there is no point …

@ Monkey
if you are not interested in discussing the questions plz do not make bogus posts regarding thread duplication…just do a “report duplicate thread” right from the homepage and you’re done…
and don’t waste your time by keeping track of duplicate threads, i guess you’re not here to do that, right? or its the other way round? OMG…!!!

Wouldn’t a static factory method defeat the private constructor?
use virtual inheritance and friend class to solve this.

class FinalLock
{
private:
FinalLock(){};
~FinalLock(){};
friend Final;
};

class Final : public virtual FinalLock
{
};

class D : public Final
{
};

Final fObj; // OK

D dObj // error!! since FinalLock constructor is private

venkat, your idea is correct, but your program is not. The reason is stated in the above compilation error.
The constructor should not be private, but the destructor can be private.

class A {
private:
// A() {};
~A() {};
friend class Final;

};

class FinalA : public virtual A
{
};

class FinalB : public virtual A
{

};

int main() {
FinalA fa;
//FinalB fb; //Error in Compile
return 0;
}

Classes:
How do you keep track of how many objects in given class exist?

use static!

I guess it can be done in the constructor.
Have to use a static int counter incremented in constructor and decremented in destructor

Apart from that ,
it even has to incremented in the copy constructor.

Define a static int to keep track of the number of objects. Increment it in each of the constructors (be sure to define the default constructor and the copy constructor). Decrement it in the destructor. Also, define a method that returns the static value.

Using static may not be working in multiple threads situation. Can you come up a better way?

Use of static is a good idea. In multiple thread situation, you can write a thread-safe code to update the value of static variable

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!