Virtual machines are the greatest thing since sliced bread and most of us techies use them daily. In this post I would be talking about using Amazon EC2 as a VM hosting platform (we as a company moved everything up there over last 4 months) and as a bonus, will describe the process of converting existing VMWare Workstation images to EC2 instances.

There are so many things Virtual Machines (VMs) help you if you’re a consultant:

• you can configure the same environment as on client site,
• run through product updates & play with new features without screwing up any physical boxes,
• be able to undo any changes (Ctrl-Z for whole server),
• run trainings on “definitely working” image,
• can configure “vanilla” demo server and use for all your “show & sell” meetings. I remember the time we had special “extra powered” laptops for that. Screwing with those laptops was severely punished (by having to reinstall everything and getting your share of mockery).

Anyways, you all know that and are surely using lots of VMs in your daily life. Using VMs leads to quite a few problems, though: lack of processing power and hard disk space.

In a small company (and in a big one as well), it grows quite painful over the time. You can do a lot with modern laptops (mine has a quad-core i7 and 16GB RAM + fast disks), but it’s really a single user story, you can’t do proper development like this. You usually have a few powerful servers that you use a VM hosts and they most of the time looks this:

Although there are probably a lot more sucker fish (usb portable hdd’s) attached to them )

And then you need “just another VM” for “another important project” and you’re running around trying to find what VM you can stop and copy to external drive, because there’s no more disk space. Rings a bell?

Or one of those big boxes collapses and all hell breaks loose.

Consulting VM usage pattern is quite different from normal infrastructure story, you usually need quite powerful machines for relatively short periods of time (development), but you have to be able to put them back online quickly if something needs investigation. Which makes Amazon pay-by-usage quite a good idea.

Enter Amazon EC2 (or any other cloud provider, to be frank)

You’ve probably heard about Amazon EC2, and if not, just read one of these links (official, wikipedia). In general, it’s a utility computing cloud where you can rent a server for quite low cost on per-hour basis. I’ve used Amazon EC2 for testing Hadoop-based OLAP implementation I’ve made for my PhD almost 5 years ago, but they’ve added Windows support a few years ago and an ability to import VMWare images.

Using Amazon EC2 pro’s:

• you can add more servers on flight, running a new one from scratch takes about 15 minutes,
• you can choose and change “instance size” for existing ones, like adding 64 Gb of RAM for a day to test out a few TM1 server issues and then reverting back to 16,
• automated VM backups and no more “part time administering a server in the closet”( after a while it’s not fun anymore and your time becomes too valuable),
• snaphots that allow you to restore VMs to a selected point in time, or just start new VMs from snapshot. Turned out to be quite useful when I branched out a copy of Datastage server to test IBM patches on and kept it running in parallel while we tested all support suggestions,
• cost reduction, cause you’re never using servers 24*7 hours in consulting, it’s more “burst-while-devevelopment-dust-while-support”.

Con’s:

• don’t expect too much from performance standpoint, Amazon CPU’s are a lot slower than “physical ones”. Although there instance types with decent allocated CPUs, but they’re not there yet in the regions we’re using now,
• network configuration issues. Each Amazon EC2 gets a dynamic ip on launch, so if you want to have a demo box with properly configured Cognos mobile stuff, you’d need an Elastic IP and a bit of configuration,
• bandwidth. Really a pain in AU. Uploading / downloading VMs can be a lot of pain. We started with Singapore (it’s our second office anyway) as region, so all uploads went through my NBN and still took a whole night per VM. There’s a Sydney region (ay!) now, haven’t tried that out yet.

Overall, after 4 months of usage, overall impression is quite positive. We’re using EC2 for a fair share of development and all training gigs (you don’t copy and leave VM after training, performance is stable, no running around with hdds).

Bonus. How to convert an existing VMWare Workstation VM to EC2?

Not a straight forward process and implies command-line manipulations (scaaary). There’s built-in VM conversion for ESX server VMDKs, but not for Workstation / Fusion ones yet.

What I did was:

1. Convert VMWare image to VHD (Microsoft VM) using StarWind or VirtualBox
2. Upload resulting VHD with standard EC2 command-line tools

Downloading back was the same process, download as VHD, import to VMWare (built in Fusion).

PS: Our Amazon machines were recently attacked by Trojans, so be mindful, it’s a hard world out there, bring band-aids antivirus. Although it still puzzles me that somebody can attack a server with dynamic ip available only 30 minutes a day with just a couple ports open )