Tag Archives: suitable

Is Big Data platform is suitable for file size 15K to 2Gb i am confirm about big files but not sure about the files of 15K size.

Is Big Data platform is suitable for file size 15K to 2Gb i am confirm about big files but not sure about the files of 15K size.

 

Your question prompts several follow-up questions:

1. Which “Big Data platform”?

2. What kind of files, and what are you planning to do with them?

3. How many files?

HDFS is not intended for small files, there’s a fair amount of overhead per file, and you wouldn’t reap many of the benefits that way. However, it might still work for you (for example, if you’ve got a relatively small number of little files, along with lots of huge files, it might be simplest to just stick the small files in HDFS and accept the wasted space, etc.).

Could you concatenate multiple files together? Or combine them some other way (e.g. if they’re log files, you might merge many of them together, sorted by timestamp). Or would it make sense to treat the filenames as keys, and the files as values, in one of the many Big Data key-value stores?

 

==

1. Hadoop + hbase
2. any file txt , logs, database backup or any media files
3. millions of file read and write simultaneously at a time with any size from kb to gbs
4. access via java api or php

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

What do you think is suitable for Low Latency, High frequency real time data. Eg. Share Market data from around the Globe

What  do you think is suitable for Low Latency, High frequency real time data. Eg. Share Market data from around the Globe

==

 

From what I know, Map Reduce is for offline data processing, I am talking about data (minor) processing and handling of real time data

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Is Quant job suitable for ladies?

Is Quant job suitable for ladies? My backgroud is BA of Math, Master of Economics, 5 years equity (stock) financial and quant analysis
and 3 year public accounting experience. But now I find real quant jobs need a phd? and C , Matlab, how long can I finish learning this two software?
Any comments are welcomed!—
—-
1. In general it is not the degree, but the quality of work ( originality, being able to deliver complex projects, persistance, etc ) + eventually the personal attitudes, that count.
2. In lieu of C you should learn C++ ( roughly speaking C AND the object oriented paradigm ). Depending on the level, can take years. Read Mark Joshi’s book, very insightful. Trying to code up some problems ( bino trees, etc ) will help. Other subtle things ( templates, boost, quantlib ) will take additional time.
3. The most important in MatLab is the “vectorized way” of programming : you can write compact code, which the MKL based MatLab numerics can evaluate quickly. It is more easy to learn than C++. You should practice it.

Read Josh’s quant interview book. If you can solve 70% of the problems, you can definitely get a job.
2. –
===
Definitely. it is just that some places are more ladies friendly than others. In my experience, there are mpre women quants in asset management companies than in trading rooms, but that is just a personal experience.
—-
Gender has nothing to do with suitability. Its your skills, your drive to learn and explore, and your capabilities to produce that count. What you need is the ability to learn anything yourself. There are plenty of online tutorials for C/C++ and free development tools. Matlab is expensive but the R languange or octave may be a suitable equivalent. Most common models and some workable strategies have been documented and published. All of this is available free on the web, including some couses on qauntitative financial engneering from major universities. Go for it.
—-
Of course women are encouraged to apply for quant roles. The quant space is becoming more and more challenging as managers require not only the quant background but also strong interpersonal skills. Let us know if you require additional information.
—-
The need for C and Matlab is just something that is needed for specific roles. It doesn’t classify the whole field. The two are prominent as most quants get exposure to them while studying and are therefore comfortable with them when they start working. C++ is most popular that C because it offers OO and a set of libraries that are helpful.

But moving away from the programming discussion and unto the “battle of the sexes”. The field is largely governed by males and there are various reasons for that; women not feeling comfortable applying because of various reasons, introverted mangers preferring to deal with male subordinates rather then ones of the opposite sex etc etc. But there is no reason why women shouldn’t be mens equals in the field.

Just for a small view my current team had precisely a 50-50 ratio between males and females before I joined and my arrival only shifted that number marginally.
—-
You don’t need a PhD but a good knowledge of quantitative finance is essential. A strong math background is necessary but rarely sufficient if you are not applying for a grad role. If looking to get into derivatives, it is best to know the main models and concepts. Two good books are Options, Futures and Other Derivatives by Hull, and Efficient Methods for Valuing Interest Rates Derivatives by Pelsser.

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant development: Algorithms suitable for FPGA?


Quant development: Algorithms suitable for FPGA?

I’m looking for some algorithms suitable for FPGA implementation to build some examples that demonstrate the approach. I’m thinking something that processes either live feed data or a large historical database. Any suggestions? I’ve already done monto-carlo

How about implementing SSL and other encryption protocols? those are pretty cpu and data intensive. not directly related to finance, but widely used none the less.

The market has moved past demoware in FPGAs. You’ll need to come up with a complete solution.

 

I have generally seen the Financial space sorely lagging in technology adoption: sometimes rightly so since they desire extremely stable solutions as it is a mission critical application, ut mostly for the lack of understanding of the same. However, there seems very little propensity to support development of new technologies , even if it gives a major performance edge, due a lack of understanding of technology at the decision making level.

As for FPGA implementations double precision FP and Linear Algera problems look at LINPACK and LAPACK benchmark performances on FPGA systems,

It is also useful of-course for non-computational pure protocol termination: Terry has a good whitepaper on the same.

 

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!