Experience with Windows HPC Server 2008 comparative to other HPC OSs
Hello! I know most high performance computing clusters use a unix/linux operating system. This is the traditional operating systems for clusters (can be lightweight, fast, and many applications are developed for it). However, I might have the opportunity to develop a new HPC application on a cluster, with the need to add nodes as the problem sizes get larger (scaling is imperative) and having real-time interaction with the computation. My application is similar to a multi-player game with bots, but each player in the game performs many compute-intensive operations on the other players in their vicinity. I will also need to make use of GPUs on each node. I am wondering if anyone had any experience using Windows HPC Server 2008 and if there are any “gotchas” that come with the OS. Any pros/cons with the OS? Would you recommend it over the Cray OS or other Linux OS that comes with systems from HPC vendors?
I know Linux/Windows is a touchy subject, but I want to ensure that I am building the code on the right architecture to begin with. Please keep the commentary professional (hence the reason I ask here instead of on a site like Slashdot). Thanks!
I have helped setup Windows 2008 clusters in the past for testing our scheduler at work. The main “gotchas” came with the initial installation of the OS and applying the service packs in order. The administration of the cluster is more difficult then a standard Linux OS as the logging in the scheduler is very cryptic. As far as scaling I believe Microsoft last told me it can scale to up to 500 nodes but that was over a year ago so improvements may have been made since then.
The one thing I do find really interesting is its ability to cycle harvest natively with PC’s running Windows 7. I have yet to meet someone who has done this but I think it would be great for certain types of serial jobs.
I work with both Linux-based and Windows-based clusters/schedulers to test our product. I’d want to know how much “other” Microsoft stuff you have, as Server 2008 / HPC Server can fit in really nicely if you have existing Microsoft infrastructure in place already and are somewhat familiar with “the Microsoft way” of doing things. If you can leverage existing patching, authentication and deployment stuff, rolling out HPC Server can be very straightforward. The installation process is very similar to any other MS software installation, they do support unattended options (they are documented), and the scheduler itself has a setup wizard that steps you through pretty much everything you need to do, and even has diagnostics that you can run to make sure things are set up correctly. Out of all the schedulers I work with, it’s by far the easiest one to set up. They also have any number of PowerScript utilities if you really want to dive deep.
The only reliable “gotcha” I’ve seen is when it comes to networking. NFS much easier on file servers than CIFS/SMB, and therefore you can get better performance and more nodes. CIFS/SMB will get the job done, but you’ll likely see more networking traffic to get the same results you would with NFS, so that might be one of the “gotchas” you are looking for.
If you are worried about stability versus Linux, I’ve been generally impressed with how MS has improved in that regard steadily since the days of NT4. Nowadays when we see servers start to have uptime problems we start looking for hardware problems first rather than OS problems.
Being in a Windows Environment, we setup two flavours of software, one running linux HPC and one Running Windows HPC. We saw a significant difference in the ease of setup, hardware standardization, etc. We most likely saw this advantage because we are in a windows shop, but we have been using Windows HPC for quite some time and it works great. The new features in 2008 R2 SP1 are significant and our users of the software really liked the fact that they could use unused compute cycles on windows 6 machines and our standby backup servers. Also, Microsoft was very accomidating with assisting
Our current infrastructure is Microsoft, but we also run Linux servers. However, we are uncertain as to how this system will be deployed/hosted and I wanted to grab some opinions on the differences in the two platforms. I’ll need to dig deeper and actually communicate with various hardware and software vendors to learn of any performance pitfalls with the platforms at a very technical level.
Thanks for the feedback!
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!