Don't forget the basics

I have a few rules which I try to abide by in my working life, one of which is 'Keep it simple, stupid!'. There is a great deal to be said for cutting out overcomplicated and unnecessary configuration, particularly when it comes to troubleshooting an existing system.
Don't get me wrong, I enjoy the cut and thrust of deep technical development and configuration as much as the next (slightly geeky) guy, but experience has shown that the business will thank you for a well designed, simple yet efficient solution, and that there are diminishing returns but increased risks as complexity increases.
This isn't to say that some services don't need in depth configuration, more that it's important to know when to stop.

Do you really need it?

The world of HPC is studded with cutting edge technology, all of it at a price. The question is do you need it?
This sounds like a simple issue, but in reality it's often quite a challenge to determine whether something will provide increased performance for your (or your users) jobs.
To cut a long story short, the most useful thing you can do to come up with the answer is to determine the requirements of your problem space. Are your tasks particularly reliant on disk IO? Do you have a sensitivity to latency? Maybe your cluster is used to run lots of single processor jobs and does not require high MPI performance at all. Once you know this it's easier to cut through the admittedly shiny technology and focus on what would be of most benefit to your environment.

Damn there are some clever people out there

The technical computing world is populated with some scarily intelligent people, seemingly brought up from an early age on complex algorithms and deep mathematical theory. Unfortunately I am not, and will never be, one of these people no matter how hard I try, so I have to look for other ways to make myself useful.
Luckily, skills developed in the enterprise world are also useful when managing Windows HPC Server, as a lot of the building blocks are common. Aside from a technical tool belt armed with AD, SQL Server, WDS, DNS etc etc, some of the softer skills can be very useful too. Knowledge and experience of support processes, project management, incident resolution, dealing with customers, and service delivery are all as applicable in the HPC world as any other environment.
After all, your carefully constructed and configured cluster is providing a service to your customers which is often as critical as any enterprise offering.

The Hit Parade

So, I've been thinking about some stuff about my time as a Windows HPC Sysadmin which I'd like to share, & the more I think, the more top level bullet points I come up with. I thought I'd kick off this blog with a couple of top tens. The first are somewhat abstract topics, the second specific to Windows HPC Server. I'll then follow it up with some more detailed blog posts which dive a little deeper into each of them.

So, without further ado, I give to you:
The Ballpark, and The Strike Zone

The Ballpark

Increase your generic High Performance Computing smarts.
Many new Windows HPC Sysadmins will be heading in from the Enterprise, having expertise in Microsoft platforms but relatively little knowledge of HPC.

Trust the product group.
You know, no-one's perfect, but the HPC product group come pretty close.

High Performance is relative.
You don't necessarily need a multi-thousand core set up to produce heroic results.

You get out what you put in.
Contributing to the Windows HPC forum has been a very useful process.

Damn there are some clever people out there.
Sometimes I just feel inadequate, desperately trying to wrap my feeble brain around complex problems discussed by some of my peers.

Do you really need it?
There's a bunch of people out there intent on selling you a bunch of kit, but is it right for you?

The methodical process of troubleshooting.
My advice - start from the bottom & work up... or start at the top and work down.

Don't forget the basics.
There's pleasure to be found in the simple things in life. Or, if you want to be blunt, keep it simple, stupid!

The business is (nearly) always right.
I find there's direct correlation between good business reporting and being left alone to get on with work.

Try to get to SuperComputing.
No, really, it's a great show, with great people.


The Strike Zone

Know your underlying infrastructure.
You only need to look at the Top 500 results to know that alot of HPC kit does not currently run Windows HPC Server. This means that many Windows HPC Sysadmins will be transferring their expertise from Linux (or other OS) based systems.

It's all in the name
How does Windows HPC handle name resolution?

Digging through the versions
SP1? Well, yes, but SP1 of what? And R2? Is that beta or RTM? So you want to install what on where now?

Performance enhancing shrugs
Well, it seems to run OK, but is OK good enough?

The database interloper
SQL Server? In every Windows HPC Server deployment? Surely not.

PowerShell is powerful
OK it might sound like an obvious statement, but it's true! At the risk of offending those not listed I have some favourite HPC PowerShell Commandlets I'd like to share.

Network topology - choices choices
RRAS, dedicated router, managed firewall and the like.

Monitor Lizard, the thinking person's test rig
If you've ever tried to compile HPL to run on Windows HPC you'll absolutely love Lizard. Performance results aside, it's a great starting point for cluster verification.

Node deployment
If one person with one DVD takes 2 hours to build one compute node, how long would it take that person to build a tree house?

A job a day keeps the user at bay
Become one with your users, know what they know, type what they type.