Tag Archives: dependency

Expert of Linux dependency Hell with their package manager

After I sent this,

Linux Mint Is Still the Leading Desktop Distribution

I got this reply from a newsletter subscriber: (thanks to this person sending this)

One potential problem with Linux Mint, is that you can still experience dependency Hell.  Its default package manager, like most linuces, cannot allow two versions of any library or software to co-exist on the same computer at the same time.  Generally, but reportedly not always, such package managers will also not allow you to install software into a directory of your choosing; as is almost always the case on Windows.  You may be able to use the second-generation package manager LinuxBrew on Mint and other Linuces; and that may be a very good idea if it works, but I make no guarantees.  The main Distros built around a second-generation package manager are Gobo Linux, and NixOS.  NixOS uses a declarative language package manager (for better or worse), and has a much larger user base, but it is pretty new.  In the past, it had a problem with uninstalling old versions of packages, but I think that is much improved by now.  Gobo Linux does some fancy footwork so that it has a Mac-like file structure.  It can download a file and all its dependencies into a single directory, and you can make it anywhere you like.  Some tech pundits complain that software doesn’t reside in typical directories for Unix, such as /bin and /usr, but this is the way things work on the Mac, and this feature seems pretty popular with users.

Ubuntu-based derivatives have better driver support, so maybe if LinuxBrew works well for you, this might be the best route.  I had lots of problems with dependency Hell in the past, and quit using linuces.  If I were to go back, I would either use LinuxBrew, or use NixOS or Gobo Linux.  Of course, there are Hackintoshes, if you want to go that route, but you probably don’t want to blog about that.
—-
Arch too is usually managed with pacman, which, like most Linux package managers, is subject to dependency Hell.  I.e., if you want to install two different programs that use e.g. two different versions of a library like zlib to read .zip files, when you go to install the second program, it will offer to uninstall the old zlib, and update to the newer version.  It will also be offering to uninstall the program that used the old version of the zlib, and will not allow both to be on the same system at the same time.  This is a tame hypothetical example of the kind of dependency Hell I’ve experienced.  If you, like I, might prefer to put audio apps or games in their own separate directory, so as to keep an eye on the size of all audio apps or games, it is either very hard, or impossible.  I have a background in computer programming myself (surprise), and believe this above all else, is what has hampered the large-scale adoption of Unix/Linux.  While I’ve only posted this point of view one other place, I’d really like to see second-generation package managers eclipse the old ones.

Manjaor Linux

I was thinking it probably used pacman, but now notice that Manjaro uses pacmac, a package manager designed for the Mac, but is not a direct port of pacman.  I don’t find a lot of documentation on it online.  Not a good sign, but perhaps it has a good man page.  My recommendation is to try to determine if pacmac can install to a specified directory, and tolerate different versions of libraries to co-exist at the same time.  To be fair, many users don’t need anything but the latest versions.  If the latest version of some software comes up with a bug that makes it unusable, you will probably want to find an old version, or a beta.  If you do that, if ANYTHING else use ANY library in common with it, hello dependency Hell.  If you’ve ever been caught in that, you’ll know it can be a session frustrating enough to make you want to pull out your hair.  I consider these landmines waiting to happen.
One time (if memory serves, or else I’m just confusing it with a time I tried to uninstall a media player, but I think that was a second time such a thing happened)  tried to install a different version of a program on Ubuntu, and since they have taken the insipid step of making it so that if you e.g. uninstall a media player, game, or ANYTHING it comes with, you wind up uninstalling the whole GUI desktop.  I wound up with a bash prompt, and NO desktop or desktop apps installed, whatsoever.  I think it was a command line package I was trying to downgrade.  I’m trying to warn you off of such nightmares.  Fortunately, Arch does not have such a ridiculous setup, but there’s still really still almost no end to the chaos that a conventional package manager can cause if you try to install another version.  This NEVER happensOne potential problem with Linux Mint, is that you can still experience dependency Hell.  Its default package manager, like most linuces, cannot allow two versions of any library or software to co-exist on the same computer at the same time. on Mac or Windows.  Mac and Windows are by far more popular.  Go figure.
PS: I was told you can install a package to a user-specified directory in CentOS.  I looked at it briefly, and it didn’t seem very easy.  I think there is a way to edit an RPM package to install in a user-specified folder, also; FWIW.
PPS: The nick I use on technical forums the most often is CodeLurker, if you want to know who to have it signed by.

Red hat Linux vs Fedora

They share a lot of code, apparently.  Red Hat Enterprise, if memory serves, is the more stable of the two, being for businesses, but at the cost of a slower release schedule.  Fedora would be for “consumers.”  CentOS, like Red Hat, uses RPM (Redhat Package Manager) archives.  It is probably possible to install to different directories in Red Hat, like CentOS; but CentOS uses Yum to install RPM packages.  I’ve been told by a system admin friend you can install to different directoris in Yum, so maybe Red Hat too.  I tried to look up how to do it in Yum, but it didn’t look easy (to me, anyway).  I think all are susceptible to dependency Hell.
For automated trading, I’d choose Red Hat over Fedora, for the sake of stability.  While Fedora might have more drivers, and should have more recently updated software, Red Hat is Big Blue’s (IBM’s) flagship.  However, I’d hold out for NixOS, Gobo Linux, or something that uses a second generation package manager, such as Nix package manager, the Guix package manager, “based loosely on Nix,” or LinuxBrew.
Note that OS independent package managers such as the three 2nd gen ones I listed might run on any distro.  I forgot to mention that most Linuces are FAR more secure than ANY version of Windows.  First, Windows admits tons of hacks all the time, at hackfests.  Second, most system security breaker hackers (I don’t want to dignify them by just calling them “hackers”), don’t bother with Linux/Unix, because it has so few users.  Third, it just seems to have much stronger security.  I imagine Big Blue is pretty quick to eliminate security vulnerabilities when it finds them.  When I looked at more secure distros, the trade-off there seemed to be fewer drivers for these more exotic distros – but then, it doesn’t really matter if you can do what you want on them.  This would me more true for a trading system than for general use; IMHO.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

Why I don’t like 3rd party dependency or service for automated trading

Why I don’t like 3rd party dependency or service for automated trading

I don’t believe in 3rd party system anymore. These are usually put out by people with little knowledge in the markets in todays automated programming world. That is just my view but I am skeptical for a lot of it. This was sent in by a new Quant Elite Member

Join my FREE newsletter to see how I planto build out my automated trading system you can depend on 

http://sceeto.com/

http://www.stockmarketfunding.com/

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!

estimate Fill Ratio dependency on the Latency in orders execution for a HF strategy, what would you do ?

aIf you need to estimate Fill Ratio dependency on the Latency in orders execution for a HF strategy, what would you do ?

I run back-testing on e-mini S&P500 and the software takes 100% fill ratio (for my limit orders) by default which is unrealistic for a HFT. The fill ratio should depend on the speed of execution and the size of orders, but how it can be estimated before real-money tests ?

How do you simulate a fill?

For simulations based on trades:
* You get a complete fill on your limit order when your limit price is traded
* You get a complete fill at your limit price as soon as a better price is traded
* You get a partial fill at your limit price as soon as the a better price is traded with the fill size the traded size.

Or do you have market depth data you can use?

est with a 1 lot , only assume fills when the price is breached. If your system still works trade live.

et’s say, I have Omega TradeStation and I don’t have market depth data. So the only situation when I can be sure in a full fill of the limit order is one when the order was generated “relatively” (to be defined) long before the trade occured (so it was actually in a queue) + the next price after the trade was better than the price of the trade (which means the limit orders of that price were executed 100%). In all other cases the order – in reality – might be executed partially or not executed at all due to too big latency in the route … Is there a chance to find a kind of 3D empirical graph linking actual latency, order size and fill ratio? Yet, there is also a time of the session parameter

Great! This is exactly the way I usually do it. I run optimization of the limit order price and then take the number of trades (frequency) from the “more distant limit price” set of parameters while the profit per trade – from the “less distant limit price” set of parameters. Multiplication gives an adjusted TotalNetProfit. Still the problem remains if there is a need to increase the size (for instance, for a business plan) or to move into HFT area. When frequency is high the role of latency becomes too uncertain/important.

You might like this paper:

Order Book Simulator and Optimal Liquidation Strategies.
Su Chen, Chen Hu, Yijia Zhou
http://www.stanford.edu/class/msande444/2010/2010p8.pdf

If you enjoy modeling, you could define a limit order flow model, simulate the orderflow and the orderbook, and tweak parameters untill the limit data you have and the simulated equivalent match.

If you go low latency then Tradestation is not the best tool.

ight you are. I used to work with it on slow strategies. It’s like old shoes – they do not work any more, but you still love them.

where did you get those rules? Any market practitioner would reason that you cannot blindly assume near-low-latent limit order execution. @Alexei, yes 100% (absolute) fill is unrealistic for HFT. The low-latency objective seeks to reduce slippage for flash traders. Flash traders are not seeking limit orders. Understand the assumptions in HFT. Separately, the fill-ratio is demand-supply, volatility dependent. Speed of the LIMIT ORDER’s execution would become increasingly significant in a increasingly active/volatile market, where the market is deviating from idle/low volume&volatility to active/high volume&volatility, esp. if you consider market makers moving the the price away from its center during low volume (less) and during high volume (more) periods. If I am mistaken/wrong, please provide citations / papers :: links.

HOW DO YOU START A PROFITABLE TRADING BUSINESS? Read more NOW >>>

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!