Project Scope
A company I recently worked for had 10 physical servers for a user base of around 50 staff. The company was once part of a much larger enterprise and its staff were used to using enterprise class networks and services in-house including:
- Microsoft Exchange 2003 Server
- File and Print Services
- Network Security (AV, Email Security etc.)
- 2 x Active Directory Domain Controllers
- McAfee ePolicy Orchestrator and WSUS Deployment Server
- Intranet Server
- Windows Storage Server based NAS for Backup/Restore
- VPN server
- ISA Server
- An engineering database server
Most of the servers the business were running were old and approaching the end of their useful life and so a project was started to determine the best way of providing following key deliverables:
- Increase the performance of key services
- Increase the available storage to the business
- Reduce licensing and running costs
- Introduce a new test environment for server/application patching
- Reduce the administrative maintenance window
- Enhance the recoverability of servers/services and protect against single point of failure
The first stage of the project involved performing a thorough audit of the network and server machine utilisation (CPU, RAM, Disk etc.). This gave us the following information:
- A baseline of server/service performance and end user experience.
- An analysis of existing infrastructure to see if any if it can be re-used.
- An insight into whether virtualisation was appropriate for this business and whether any of the services could not be virtualised.
- A platform on which we could specify an appropriate solution.
The next step was to perform some appropriate research into the software and hardware necessary to fulfil the project goals.
Research of an appropriate solution
At the time the project was initiated Microsoft HyperV and VMware ESX (now vSphere) product offerings were compared against each other in terms of features and price. Two key features were missing from Microsoft’s offering that meant that it couldn’t easily meet the design brief. The first was High Availability (HA) and the second was vMotion (live migration of running virtual servers between physical host servers). HA and vMotion allowed the solution to meet goals 5 and 6 of the project scope. On this basis VMware was chosen as the software virtualisation platform.
In order for HA and vMotion to be implemented a shared storage solution would be required to host the virtualised files. Dell EqualLogic are one of the market leaders for Storage Area Network devices and we were able to secure a great deal on the PS6000E with 16x500GB SATA drives. The storage solution gave the business a total of 5.2TB of useable storage with the redundancy features of RAID 50 (which allows up to 3 individual drive failures) and dual disk controllers, and dual power supplies. On the software side of the EqualLogic unit, Dell provides excellent snapshotting features which allow snapshots of running virtual servers to be taken on a schedule of an administrator’s choosing. Snapshotting would prove to be a useful recovery tool for the business.
The existing Gigabit network at the business was based upon HP Procurve layer 3 switches. These were more than adequate to be re-used to provide the network backbone for the solution. To prioritise certain types of data on the network and to compliment the existing VoIP system in place at the business, additional VLANs were configured to give higher priority to the voice and storage data packets on the network.
The solution required 2 physical host servers and a vendor needed to be chosen for them. The business had historically standardised on HP equipment and so 2x HP Proliant DL360 G5 machines were chosen. Each machine had 32GB RAM, 2 x Intel E5505 Quad Core processors and 2x 72GB SAS HDD in RAID 1 configuration. Since the EqualLogic SAN would provide the storage for the virtual machines, the internal storage for the host servers only needed enough space for the VMware ESX software. Each server had 2x internal network ports and another 2x 4 port network cards were added to the spare PCI Express slots to give a total of 10 network ports per server. This provided another resiliency benefit to the design so that enough resources were present in each host server so that all of the virtual machines could be run without performance penalty on one host whilst maintenance was carried out on the other host.
Finally software and licensing implications of the solution were researched. It was found that Microsoft ISA 2004 server was not supported at the time on virtualised platforms and so that server was excluded from the scope of the project. Regarding Windows server licensing, on VMware platforms the Microsoft Server 200x Enterprise license allowed the installation of 4 instances of the server software with one caveat. The physical host server counted as one license regardless of whether it ran the OS or not. This therefore allowed for 3 virtual instances of Windows Server Enterprise per license bought.
On the VMware side, a license was available that provided all of the VMware suite of tools and features for up to 3 physical hosts with 2 x CPU’s in each host.
Deployment and Configuration
The work started with VLAN reconfiguration of the existing switches. An existing HP Procurve 2900-48 became the Layer 3 routing switch at the core of the network, with other Procurve 2900-24 port switches providing additional port density. Voice, Data, Storage, Vmotion and Test VLAN’s were configured on each of the switches and port assignments and routing were tested.
The EqualLogic SAN was then set up so that it’s network ports were configured on the Storage VLAN. In addition for each controller card half of the network ports were configured to one switch and half to another. This is recommended practice to build in redundancy against a network switch failure.
Next the host servers were installed and set-up with ESX. VirtualCenter was installed on a physical server and the licensing activated. The virtual networking was configured so that virtual NIC’s used the appropriately assigned VLANs. Storage controllers were then configured, and storage was then provisioned first on the EqualLogic and then on Vmware. The storage and hosts were then ready for virtual machines.
HA and Vmotion were set up and a test physical to virtual migration (P2V) was performed using VMware converter. This proved successful and vMotion and HA testing took place successfully. The VMware converter tools were so good at performing P2V migrations that all of the existing physical servers were migrated this way into virtual machines. The conversion process was managed to minimise the downtime of the services to end-users. The virtual servers were then allocated more RAM, CPU, and storage as required to increase performance and any remnants of physical hardware device drivers and registry entries were removed from the virtual servers. VMWare DRS rules were set up to manage performance of the virtual machines between the hosts and priorities were assigned so that the most important machines had the highest priority on available resources.
Finally end-user testing took place to ensure that all virtualised services performed as expected.
Post-Deployment
Looking at the project goals one by one:
- Increase the performance of key services
Performance was noticeably quicker for end users as services that were previously CPU limited running on Pentium 3 class hardware with 1 or 2GB RAM were now running on Intel Quad Core processors with more available RAM. For those services that were not pushing even Pentium 3 based hardware there was no noticeable speed improvements save in the loading time of the Windows OS.
- Increase the available storage to the business
With the EqualLogic SAN in place storage was easily provisioned on demand. Total storage available to the business was approximately 5 times what was available previously.
- Reduce licensing and running costs
The main savings made were in power and cooling requirements due to the decrease in the number of physical servers within the building. There were also savings in hardware maintenance costs since the number of servers in total on maintenance contract was substantially reduced. There were additional savings made with license costs for Windows Server and subsequently additional savings have been made when the business upgraded some of its Windows 2003 based licenses to Windows 2008 R2 giving it ‘spare’ licenses it could use if needed.
- Introduce a new test environment for server/application patching
This benefit was of importance to the business given that before virtualisation a test environment wasn’t affordable or realistic. Clones were made within VMware of servers on demand and patch and application testing could take place within that isolated environment to determine the effect of those patches or applications on the server totally risk free.
- Reduce the administrative maintenance window
Whilst it is impossible to reduce the administrative maintenance window to zero there were benefits to uptime as a result of the virtualisation project. There were also benefits associated with the EqualLogic snapshotting tools that enabled a fast rollback of virtual machines to an earlier point in time should something go awry with a Windows installation.
- Enhance the recoverability of servers/services and protect against single point of failure
The virtualised solution provided excellent benefits in this regard. The EqualLogic SAN took snapshots of each server on a schedule approved by the administrator. For the mission critical servers this was every 15 mins and if needed the rollback process took less than 3 minutes to recover a virtual machine. The snapshotting regime became an additional tool in the backup/restore regime of the business.
However whilst this solution provided a great deal of resiliency it did not fully satisfy the requirement of protecting against a single point of failure. To do so would have required a redundant SAN which would automatically replicate changes to logical volumes to the secondary SAN device. An additional consideration in order to protect against a potential disaster at the premises was that the redundant SAN would have to be located sufficiently far away from the primary SAN unit. The budget wasn’t available to do this at the time but was completed in the following financial year as a phase 2 project.
Conclusion
Server consolidation using Virtualisation technologies can provide businesses with a number of benefits and features that a purely physical environment cannot. These include a large degree of flexibility in providing on demand computing resources, lower licensing and running costs, and an additional means of disaster recovery. However, for a fully redundant virtualised environment the costs of entry are significant, however as technology develops and prices fall, the benefits listed in this case study can be enjoyed by more and more businesses.