Tag Archives: transform

Skills that transform you into skill algo trading

Skills that transform you into skill algo trading

TBH, this is pretty good for summaries

https://www.experfy.com/blog/skills-that-can-transform-you-into-skilled-algorithmic-traders

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Machine learning will transform investment management

Machine learning will transform investment management
According to this article, the role of a quant will radically change in coming years

Advances in machine learning will allow further automation of tasks, including feature discovery, algorithm selection and even the optimization of trading code that implements a signal. With these newer methods, humans can spend their time creating frameworks and obtaining new data sets. Automated methods also reduce the number of quants needed to run a firm, which, given the high cost of salaries, is increasingly important as well.

The tools that enable automated strategy discoveries will also enable customized solutions. This is because the criteria for a successful strategy can be specified upfront in the framework. For instance, one approach could search for a strategy with a specified maximum deviation from a classic index, such as U.S. small-cap value, with as much additional alpha as possible. This type of customization could lead to a new class of retail investment products.

http://www.pionline.com/article/20170320/ONLINE/170319958/machine-learning-will-transform-investment-management

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to transform to a fully algo trading company

How to transform to a fully automated trading company

A while back one of the last remaining Algorithm trading software vendors did a talk about transitioning into an automated trading company This video is over 30 minutes long at one the most prestigious Quant events on the planet.

 

Go here to see this presentation

 

As you know, I’ve spent many many years researching the best ways to do exactly this. There’s many different forms of doing it or applying it but the question comes down to which is the most  lucrative.

 

As I have been revisiting the videos of the Deltix HFT platform, I’m starting to see some of the huge advantages that it brings. I would highly encourage you to watch them as I have listed a few videos in the past couple of emails and blog postings. They will bring so many advantages to your capabilities of getting much higher bank with very little modification.

 

So I continue to apply my software architecture experience to building out a hopeful but highly advantageous trading system that will work for me. Once I get things rolling, I will put small software components and nuggets for my Quant Elite members.

 

This is one of the many advantages I bring for them let alone the course that I’m doing right now on how to build out an exact basic Algorthm/automated trading system.

 

You can see the major benefits and details of this course series here:

If you are interested in joining, here are all the pricing options:

MONTHLY: $97/MONTH: Click here

6 BONUS MONTH FREE Annual: Click here

BIGGEST SAVINGS with 24 BONUS months: Click here

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

How to transform to a fully automated trading company

How to transform to a fully automated trading company

Here is a video on how to do it from one of the most respected quant conferences on the planet

Join my FREE newsletter to see how we transform ourself to a fully automated trading firm

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

MATLAB to transform your TRADING ideas into an algo

MATLAB to transform your TRADING ideas into an algo

These are some of the advanced ideas you can get out of this thing:

Use MATLAB to transform your ideas into algorithms
The MATLAB environment lets you explore multiple ideas and approaches. The high-level language automatically manages low-level programming details such as memory management and variable typing, letting you focus on what you want the algorithm to do.

MATLAB add-on products provide built-in algorithms for signal processing and communications, image and video processing, control systems, and many other domains. By combining these algorithms with your own, you can build complex programs and applications.

Sonar and Echosounder Data Analysis Software Analyzing High Speed Video
Developing and Deploying Sonar and Echosounder Data Analysis Software
Read about how SonarScope used MATLAB and 6 different toolboxes, including Control System Toolbox and Signal Processing Toolbox to allow them to develop algorithms, visualize results, and refine the algorithms in an iterative cycle. Read more.
Analyzing High Speed Video Images
This article goes through a step-by-step process of how Alkermes used MATLAB and Image Processing Toolbox to develop a video capture and analysis procedure. Image analysis algorithms were designed, prototyped, tested, and improved upon repeatedly in the interactive MATLAB environment. Read now.
Leverage the built-in algorithms in MATLAB and add-on products to develop your own application. Request a trial and get started!

Join my FREE newsletter tif you want to learn how to apply Matlab into your trading

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Best way to transform a dataset? The data failed to meet normality – I have tried a log, log10, and a sqrt transformation.

I am trying to find the best way to transform a dataset. The data failed to meet normality – I have tried a log, log10, and a sqrt transformation. They did not fix the problem. which other options?

This article looks interesting. http://www.isixsigma.com/tools-templates/normality/tips-recognizing-and-transforming-non-normal-data/ You can also look at papers dealing with non-normal data transformation.

Can you send your dataset, I would like to see as how it can be done?

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant analytic: Best method to transform a continuous variable to categorical variable in order to build a logistic regression model?

Quant analytic: Best method to transform a continuous variable to categorical variable in order to build a logistic regression model?

 

==

Which logistic regression model do you intend to use? If binary logistic: just decide on a cut-point which separates the two categories. In SPSS, you can use the recode method which is available on the Transform menu, or do it via syntax.
If you create more than two categories, you might look at ordinal regression, which is an extension of the binary logistic model, but quite challenging to interpret. I assume your categories would be ordered, so in a multinomial logistic model you would lose part of the information contained in the data.

==

 

I would urge caution and recommend you reconsider whether you want to really want to “bin” your continuous outcome variable.

Logistic regression is best applied when the two outcomes reflect distinct states (for example, has diabetes vs. does not have diabetes). If you took a continuous variable, like income, and binned it to “over $40k” and “$40k or less” you really don’t have distinct states … the difference between $39,999 and $40,001 is trivial.

If you are struggling with a skewed outcome variable, I recommend you consider these two alternatives before resorting to binning it:
(1) Use a generalized linear model and select an appropriate distribution (Poisson and Gamma are quite popular); or
(2) Try transforming your outcome variable (such as a log transformation) to see if that makes it “more normal”.

 

==

 

==

You can generate a seq of cut-off points and then try to separate the continuous data to binary using the cut-off. Based on each logistic regression, calculate the AUC. Find the highest AUC and the corresponding cut-off. I think that cut-off may be the optimal one to classify your data into binary.

 

==
i don’t understand how to use a generalized linear model sutch us poisson or Gamma to bin continuous variable can you give a simple example if you want.

==

 

I was suggesting consider using a generalized linear model instead of binning — not as a method to create bins. Sorry for the misunderstanding.

 

==

 

I wouldn’t recommend doing that. Why would you want to lose richness of data that have been collected using a ratio scale by downgrading it to data using a categorical scale. It might be better to run linear or non-linear regression and thus retain the robustness of the data you’ve collected. I’d recommend looking at various bivariate scatter plots (two-dimensional plots between the dependent variable and each of the independent variables, one by one), as the first step, to understand the nature of relationship, and then chose an appropriate regression model accordingly. … But if for any reason, you must want to change the dependent variable to categorical scale, you could just follow a simple step. Classify the independent variable data into intervals, examine the frequency distribution to look for distinct concentrations or groupings. Choose cut-off points as appropriate and combine the intervals into groups accordingly. Assign a value to each group, and what you now have is a categorical scale. Hope this helps.

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant analytics: Can someone help me with the best method to transform a continuous variable to categorical variable

Quant analytics: Can someone help me with the best method to transform a continuous variable to categorical variable

 

Which logistic regression model do you intend to use? If binary logistic: just decide on a cut-point which separates the two categories. In SPSS, you can use the recode method which is available on the Transform menu, or do it via syntax.

If you create more than two categories, you might look at ordinal regression, which is an extension of the binary logistic model, but quite challenging to interpret. I assume your categories would be ordered, so

==

 

 

I would urge caution and recommend you reconsider whether you want to really want to “bin” your continuous outcome variable.

 

Logistic regression is best applied when the two outcomes reflect distinct states (for example, has diabetes vs. does not have diabetes). If you took a continuous variable, like income, and binned it to “over $40k” and “$40k or less” you really don’t have distinct states … the difference between $39,999 and $40,001 is trivial.

 

If you are struggling with a skewed outcome variable, I recommend you consider these two alternatives before resorting to binning it:

(1) Use a generalized linear model and select an appropriate distribution (Poisson and Gamma are quite popular); or

(2) Try transforming your outcome variable (such as a log transformation) to see if that makes it “more normal”.

 

==

Thank you all for your responses. Its very helpful

 

==

You can generate a seq of cut-off points and then try to separate the continuous data to binary using the cut-off. Based on each logistic regression, calculate the AUC. Find the highest AUC and the corresponding cut-off. I think that cut-off may be the optimal one to classify your data into binary.

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant analytics: Transform a distribution to normal

Quant analytics: Transform a distribution to normal

A part from the box-cox transformation, are there other methods for transforming a distribution to normal?

I interpret the problem you posed as
X= your data set (a column of numbers)
You are looking for a function f(x) such that Y = f(x) ~ normal( some mean, some sd)

Box-Cox is one possibility – however there are many other possibilities depending on what is the distribution on your X

For example –
1) if X~Lognormal distribution, then log(x) ~Normal
2) If your X is like correlation coefficients (ranges between -1,1) – Fisher’s transform will convert it to normal – f(r) = 0.15 * ln( (1+r) / (1-r) )
3) In fact, based on theory of probability distributions – more bizzare choices of X and f(x) can be constructed.
– For an applied Statistician – you may need to focus on what is your original distribution of X and go from there.

For crude purpose you can substract the sample mean & divide by sample s.d.The resulting data will be approx Normal(0,1) for moderately large sample size.

If you’re working in one dimension, you can always — given a value drawn from your distribution — calculate the fraction of values lower than this value (i.e. get the value of the cumulative function of your distribution) and then use the inverse of the cumulative normal distribution to look up the transformed value. The transformed values will correspond to a normal distribution by construction.

Maybe you could tell us *why* you want to transform your distribution into a normal one ?

 

Is it the only way to go that you need to let your data follow a normal distribution, say after any transformation process? In case it is impossible to achieve this goal, there are number of methods available in literature can handle non-normal data.

Why? And I might add, it would be useless unless you could transform the results obtained after using the normal back to the proper domain.

Transformation depends on the existing distribution. Statistician performs diagnosis before treatment as a doctor performs a diagnosis before treatment. You can try a series of transformations and examine the test Kolmogorov – Smirnov If the data close to a normal distribution. Choose the best transformation. Besides there are methods of non-parametric tests with powerful statistical distributions are not normal.
If we assume that you examine the days of hospitalization in – patients. You’ll find that a large population with 0 days then? Normal distribution of days of hospitalization. In this case the solution is not a transformation but the analysis in two separate phases. 0-1 as a function of logistics and then treated normal function during hospitalization. In short – there must be a first diagnosis of the underlying data is correct before the solution.

Stephen Few has written 3 that are very helpful – Show Me The Numbers is one. The others are at work & I’m not sure of the names.

There was an interesting talk at SAS Global Forum by Walter Stroup of the University of Nebraska (one of the authors of SAS Sytem for Mixed Models). Now that we have tools like generalized linear mixed models in which we can use link functions to many different non-normal distributions, he argued that it’s less important to limit ourselves to models that assume residuals are normally distributed.

Link to Dr. Stroup’s paper:
http://support.sas.com/resources/papers/proceedings11/349-2011.pdf

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Quant analytics: How to Transform a distribution to normal

Quant analytics: How to Transform a distribution to normal

A part from the box-cox transformation, are there other methods for transforming a distribution to normal?

I interpret the problem you posed as

X= your data set (a column of numbers)

You are looking for a function f(x) such that Y = f(x) ~ normal( some mean, some sd)

 

Box-Cox is one possibility – however there are many other possibilities depending on what is the distribution on your X

 

For example –

1) if X~Lognormal distribution, then log(x) ~Normal

2) If your X is like correlation coefficients (ranges between -1,1) – Fisher’s transform will convert it to normal – f(r) = 0.15 * ln( (1+r) / (1-r) )

3) In fact, based on theory of probability distributions – more bizzare choices of X and f(x) can be constructed.

– For an applied Statistician – you may need to focus on what is your original distribution of X and go from there.

For crude purpose you can substract the sample mean & divide by sample s.d.The resulting data will be approx Normal(0,1) for moderately large sample size

If you’re working in one dimension, you can always — given a value drawn from your distribution — calculate the fraction of values lower than this value (i.e. get the value of the cumulative function of your distribution) and then use the inverse of the cumulative normal distribution to look up the transformed value. The transformed values will correspond to a normal distribution by construction.

 

Maybe you could tell us *why* you want to transform your distribution into a normal one ?

• I agree with Andre’s question. Is it the only way to go that you need to let your data follow a normal distribution, say after any transformation process? In case it is impossible to achieve this goal, there are number of methods available in literature can handle non-normal data

Why? And I might add, it would be useless unless you could transform the results obtained after using the normal back to the proper domai

ransformation depends on the existing distribution. Statistician performs diagnosis before treatment as a doctor performs a diagnosis before treatment. You can try a series of transformations and examine the test Kolmogorov – Smirnov If the data close to a normal distribution. Choose the best transformation. Besides there are methods of non-parametric tests with powerful statistical distributions are not normal.

If we assume that you examine the days of hospitalization in – patients. You’ll find that a large population with 0 days then? Normal distribution of days of hospitalization. In this case the solution is not a transformation but the analysis in two separate phases. 0-1 as a function of logistics and then treated normal function during hospitalization. In short – there must be a first diagnosis of the underlying data is correct before the solution.

 

 

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!