Why aren't databases getting migrated to VMware?

During a recent customer CTO focus group meeting, a key topic discussed was the perceived unwillingness (or should one say, inability) of many larger organizations to move their databases onto virtual servers.

The majority of databases, especially production systems continue to run on physical servers. Given the emphasis in today's economy on data center consolidation and the need to enable newer IT delivery models such as self-service and private cloud initiatives, migrating databases from expensive underutilized physical servers to shared virtual environments should be a no-brainer. However IT managers find themselves swimming upstream as soon as they broach the topic of database migrations with their application users or for that matter, their DBAs.

Migrating databases to virtual environments is difficult for several reasons - especially given their complexity and mission-critical nature, the extended time required to perform those migrations gracefully and the general lack of tolerance from application users to corresponding maintenance/outage windows. A database has hooks deep into the underlying operating system as well as the overlaying application stack. Even relatively “minor” migrations wherein both the source and target platforms and versions are exactly the same can impact database performance and stability if all the associated factors are not duly considered. Let's explore some of these factors.

A database migration in this context is defined as moving a database from a physical to a virtual environment. Whenever possible, the source and target environments retain the same operating system (OS) attributes. But frequently, those may need to be changed as well. For instance, when migrating an Oracle or DB2 database running on an IBM pSeries server with AIX 5.x to a virtual environment on VMware, the underlying operating system type has to change to a Linux flavor on the x86 platform. Accordingly, the following are the two main options for a database migration:
• The same OS on both the physical (source) and virtual (target) environments
• A change in the OS type or version in the target virtual environment.

The latter use case is obviously more complex than the former. Regardless, with both use cases, there are several factors, decision points and associated actions that need to be taken into consideration. The sequence of actions in the schematic below (Figure 1) illustrates a few of the core issues associated with such a migration for Oracle, SQL Server and DB2. (Click on the picture below to view a larger image.)


Especially as Rows 2 and 4 in Figure 1 reveal, there are specific actions that need to be taken both on the source (physical) and target (virtual) environments. Furthermore, all of these actions need to be taken in line with corporate standards and best practices including checking for change control approvals, carrying out the work within pre-defined maintenance windows, and rolling back specific actions upon failure or other environment conditions such as the maintenance window being exceeded.

All too often, IT managers and generic IT administrators (read, non-DBAs) mistakenly believe that the scope of database migrations is limited to provisioning a new target database server, copying the data contents from the source to the target and finally, re-pointing the applications to the new target environment. In this context, one needs to differentiate between a database server and a database instance. In the case of the former, one can use existing provisioning tools to set up the requisite platform and version build, patch-level, kernel parameters, file-systems and other operating system-related aspects (equating to Row 3 in Figure 1 above) and utilize standard run book automation (RBA) tools to interact with change control and ticketing mechanisms (equating to Rows 1 and 5 in Figure 1), however these provisioning and RBA tools fall short when it comes to the setup and management of much of the database internals (as shown in Rows 2 and 4 in Figure 1) – tasks that make up the core of the migration process. For instance, tools such as BMC BladeLogic, HP Opsware SAS and VMware vCenter Server can rapidly provision target database server environments, but lack the database instance and application specific context and the corresponding automation content to discern and establish crucial post-server-provisioning aspects. Conventional RBA tools such as BMC BladeLogic Orchestration Manager, HP Operations Orchestrator and VMware vCenter Orchestrator have the capability to interact with change control and ticketing systems to perform the peripheral tasks in Rows 1 and 5 in Figure 1, but lack knowledge of database internals hence requiring all the steps indicated in Rows 2 and 4 to be developed from scratch – a process that can take multiple years.

Similarly, Virtual server migration tools as VMware vCenter Converter (a.k.a. VMware P2V) and Novell PlateSpin PowerConvert that migrate a server in its entirety also don't do justice to the task at hand because a database server may comprise multiple databases/instances supporting several different applications, and the DBA may need to selectively migrate a few databases/instances at a time (based on application user and change management approvals) rather than the entire database server. P2V-like tools don't take these intra-database structures and other application-level dependencies into account. Also, these tools only deal with x86 servers running Windows and Linux. Non-x86 hardware platforms and other operating systems such as AIX and Solaris are woefully ignored.

A database instance has its own unique set of requirements and configuration options that need to be defined and managed. Take for example, an Oracle database environment. A typical mid-sized to large company (the kind that will benefit most from automation) may have dozens, even hundreds of database instances across different versions (e.g., 9i, 10g, 11g) and configurations (standalone, RAC, Data Guard, etc.). New databases have to be set up for multiple environments (production, QA, stage, etc.) and applications (OLTP, reporting, warehouse, batch, etc.). Based on the OS platform, database version, configuration, usage factors and size, different data transfer methods have to be selectively applied during a migration (e.g., RMAN duplicate, Transportable Tablespaces, Data Pump, Export/Import, etc.). Once the data is extracted, it needs to be imported into the target environment using the appropriate method(s) and reconfigured for user and application access. Structures such as stored procedures, triggers and other objects need to be transferred as well using appropriate mechanisms. External stored procedures need to be recompiled and relinked on the target. Depending on the source and target operating systems, issues such as endian formats, byte sizes and character types need to be considered. Shared libraries have to be installed and clustering related parameters have to be defined. Backup methods need to be reconfigured and rescheduled. Agents need to be reinstalled. Maintenance and other scheduled jobs have to be set up. Database links need to be re-established in the case of federated or distributed databases. Thus, several configuration, security and performance related factors need to be evaluated and appropriately addressed.

Many of these considerations have prevented DBAs from viewing automation as a reliable method to perform migrations. And performing them manually is not a realistic option since they can take up several thousands of DBA hours and carry a significant risk of performance degradation or downtime if human errors creep in. These complex issues prevent most databases from being migrated to virtual environments, in spite of the obvious cost advantages associated with virtualization. There is adequate fear prevalent amongst IT managers to keep their hands off databases and leave them running on mammoth underutilized physical servers.

That’s where database automation products such as Stratavia’s Data Palette can help. Specifically, Data Palette provides value over conventional server provisioning and RBA technologies as well as P2V-type tools in two key ways:
- due to its comprehensive database automation content, it can handle complex migration use cases out-of-the-box

- due to its ability to collect, persist and embed metadata within its automation workflows (to provide environmental awareness to those workflows), it can handle heterogeneity and ongoing changes in platform builds, versions, application attributes and usage profiles in a much more scalable manner. It allows a consistent set of migration workflows to be used across disparate environments from a central console, making the automation easy to deploy, maintain and run.

Migrating databases from physical servers to virtual machines becomes straightforward with Data Palette due to the ability of its workflow engine to span both physical and virtual environments (within a single workflow) - allowing the entire spectrum of activities related to database migration (as illustrated in Figure 1) to be automated end-to-end with the press of a button.

This automation content coupled with Data Palette’s role-based access control and Service Catalog type user interface allows senior DBAs as well as junior or offshore database operations personnel and even non-DBAs for that matter (e.g., trusted, but non-database-savvy IT personnel such as systems administrators and application project leads) to perform these migrations in self-service mode without requiring them to have knowledge of underlying database processes, and without local administrative access on the source and target servers - in a manner that meets all DBA and change control approvals.

Post-migration, a key Data Palette feature called Smart Groups™ allows the database instances to retain all of their prior monitoring, maintenance and other scheduled activity in a seamless manner without requiring any manual intervention. These advanced management capabilities and out-of-the-box automation content allows complex database physical to virtual migration projects to be completed in days, rather than weeks and months.