VMware backup refers to the process of copying data on a virtual machine (VM) within a VMware environment to prevent loss of data. VMware backup, and by extension virtual server backup, is a common challenge for backup and storage administrators. Virtual server backup refers to the copying of data that is stored on a virtual server to prevent data loss.
Conventional backup software is a straightforward approach to accomplishing VMware backup, but it can result in resource contention; the additional resources you would require to execute a backup could compromise VM performance on the server being backed up.
VMware-specific backup products that can capture point-in-time snapshots of your entire virtual machine state can be used to address any resource contention issue that arises. This would allow for a fast & complete restoration of virtual machines. However, you would typically need to restore the entire snapshot when even one file is corrupted or missing. Newer VMware-specific tools can address file-level restore.
How to Backup VMware Virtual Machines
To back up your virtualization machine (including the OS, application files, user data, and settings) you need a copy of the original folder where the VM is stored. Follow the steps below:
- Power off your virtualization machine. Leaving the VM running while copying it may result in a copy that refuses to boot.
- Locate the target folder you want to copy.
- Right click the folder and then click copy or press Ctrl+c.
- Select your preferred destination location.
- Press Ctrl+v or right click within the folder and click paste. You will see a progress bar that indicates the process is underway.
- Once the process is finished, power on the copied VM. Workstation prompts you to specify whether you have copied or moved the VM.
If you indicate that you have moved the VM locally on the hard drive, all settings will be retained. On the other hand, the “Copied it” option will warrant the generation of a new MAC and UUID address to ensure that no conflicts arise in the network.
Three Methods of VMware Data Backup and Restore
VMware backup has brought about fundamental improvements in the world of data protection such as the benefits of encapsulation and abstraction. However, some challenges still exist, including how to ensure data consistency as well as addressing the issue of excessive consumption of this technology’s physical resources.
It is tempting to think that backing up your server is as easy as backing up the underlying Virtual Machine Disk Format (VMDK) files since VMware encapsulates physical servers into just a handful of these large hard disk image files.
What’s more, backing up a virtual machine while it is running does not ensure that all the in-flight activity will be fully accounted for. This means that you risk data inconsistency, and therefore inaccurate information at the end, making the restoration unsuccessful.
The challenge of excessive resource consumption is a virtualization side effect. In fact, among the key reasons why you should virtualize your systems using VMware is to ensure that you concentrate resource consumption onto few physical servers, which reduces the amount of idle cycles suffered by most IT server infrastructures. Unfortunately, you also have to contend with the inability to have enough resources that will allow your data backups to run unhindered.
You should also know that your backups are at their most vulnerable points within VMware due to its narrow ability to handle excessive network or disk I/O. Your decision to virtualize to a physical server often hinders on the intensity of the network or disk I/O present.
Despite these issues, there are suitable methods to address them and provide you with benefits that, in some cases, could prove superior to a standard physical backup and restore. However, there is no proven best method of backing up or recovering VMware. For many administrators, the end justifies the means and what works for one may not necessarily work for another.
Method 1: Local Backup Agent Installed in each VM
In this traditional approach, a backup software agent is installed in the VM just like on a physical server. Here, data flows to the backup/recovery infrastructure, over the LAN, similarly to what happens if the agent is installed on a physical server.
The advantages of this method are:
- No procedural changes or special skills are required since the installation and configuration of the backup agent is similar to the process otherwise followed if a physical server was used.
- The restore process is also unchanged compared to a physical server file-level recovery.
- File-level recovery is possible.
- Full and incremental backups are possible.
- This method helps preserve the consistency of application data if you use specialized app-aware backup agents such as Exchange or SQL.
The disadvantages include:
- It is easy to overtax the host’s resources since all backups run over a single server.
- The backup agent fails to recognize that your servers are encapsulated into large VMDK files meaning that, from a disaster recovery standpoint, there is minimal value.
Method 2: Backup Agent Installed in ESX Service Console
In this method, you install the backup software agent within the designated ESX Service Console and back up each virtual machine’s underlying set of VMDK files.
The advantages are:
- You only require a single backup agent as opposed to an agent per VM.
- You can back up all your VMs by simply backing up the VMDK files.
- Fast image-level recovery is possible.
The disadvantages are:
- You require scripting to automate the shutdown, starting up, and snapshot of the VMs to ensure application consistency is maintained.
- No file-level recovery or incremental backups are possible.
- VMware plans to remove the Service Console from the ESX server eventually.
Method 3: VMware Consolidated Backup
VMware Consolidated Backup (VCB) refers to a backup framework that offloads VM backup off the ESX server. The configuration eliminates backup traffic from a network and frees ESX server resources for VM performance. Other features of VCB include:
- Full and incremental file-level backups on top of full image backups.
- Support for Fibre Channel, network-attached storage, and local or iSCSI storage infrastructures.
- Enables backup management from a central point.
However, VCB is not your typical backup agent, as it is essentially a command line tool consisting of several Windows-executable programs. When you add scripts, these programs can also provide a framework for other third-party products to use.
VCB suffers from some limitations including the need for a dedicated Windows proxy server and third-party backup programs. Due to the release of vSphere 4, backup vendors are turning to vStorage application programming interfaces (APIs) that have replaced VCB.
Advantages of VCB-Proxy
- You can utilize only one backup agent for all your VMs.
- You can back up all your VMs by simply backing up the VMDK files.
- Fast image recovery is possible since the process involves the streaming back of a large image file instead of seeking many small files.
- Using the VCB Proxy server for your backup process reduces the ESX server’s overhead.
- This LAN-free SAN-enabled approach should, theoretically, provide a faster backup compared to a LAN-based method.
The disadvantages are:
- Automation and ease of use of VCB-Proxy relies on the capability of your third-party backup software.
- This method is complex to implement especially if you lack a backup software integration option to simplify the process.
- You will need to install a backup software agent in your VM if you want direct file-level recovery.
- When using Windows without VSS integration, VCB’s image-level backup is crash-consistent.
- VCB does not provide a mechanism to support Windows System State backups. You may be able to successfully do a full server recovery; however, it is not guaranteed if your system was in flux during the VM snapshot process.
What is VMware vSphere?
It is a server virtualization platform that debuted in 2009 as a successor to VMware’s flagship infrastructure solution. It is a complete platform for the implementation and management of virtual machine infrastructure on a large scale.
It is also popularly referred to as a virtualized data center platform or cloud operating system. It has the capability to enable your IT department to efficiently place application workloads on a cost-effective compute resource.
The operation and architecture of a virtual environment differs from that of a traditional backup environment in that it demands specific techniques. When it comes to backing up virtual machines in this platform, you need to utilize the strengths of virtualization if you want to maximize your backup capacity and restore efficiency. Also, you cannot rely on the same principles you used in your traditional physical environment when you opt for a virtual environment.
Below are eight suitable backup practices to follow:
1. Do Not Backup Your VMs at the Guest OS Layer
With traditional servers, you usually install a backup agent on its guest operating system (OS) that your backup server contacts every time it needs to back up data. However, this method is inefficient in a virtual environment since it causes unnecessary VM resource consumption, which impacts its performance as well as that of other machines running on the host.
Instead, you should back up your data at the virtualization layer. Here, you will use image-level backups for the large .vmdk file to avoid involving the guest OS. To accomplish this, you must use a backup app that is designed to work in the virtualization environment and that can back up the machine’s virtual disk directly without the need to involve the host or guest OS. This allows you to eliminate any unnecessary resource consumption while ensuring that your VMs get the resources needed for their workloads.
2. Leverage vStorage APIs
The vStorage APIs were introduced together with vSphere to replace the VMware Consolidated Backup framework, which was released with VMware Infrastructure 3 to assist in offloading backup processing from the host. They not only enable easier access to a virtual machine’s disk file, but also contain unique features that can significantly Excellerate backup speeds, for example, the Changed Block Tracking feature.
Changed Block Tracking (CBT) refers to a feature that keeps track of any changed blocks since the last backup occurred, so a backup app only needs to query the VMkernel to get information. Moreover, this operation means that there is no longer a need for the backup application to track changed data; this allows for a quicker incremental backup. You should use apps that take advantage of the vStorage APIs due to their efficiency.
3. Never Skimp on VM Backup Resources
If you want to have the shortest backup window possible, ensure that you get adequate hardware and software for your backup server to prevent operation bottlenecks. You need to have adequate network bandwidth as well as enough memory and CPU resources.
Your backup server does more than simply moving data from the source to the target storage device. It also does data deduplication and compression to reduce your backup sets’ sizes. All the processes require a lot of memory and CPU to keep up with the data flow.
Always follows your vendor’s hardware recommendations for the server. Do not economize on your servers, since backups can slow down significantly if the server lacks adequate resources. Test out the many third-party tools available for helping with backups before selecting a suitable one to use.
4. Schedule and Test Your Backups Carefully
Virtual environment backups can strain your resources due to the shared virtualization architecture. Thus, you need to plan your backup schedule to avoid stressing a single resource. For instance, never back up many virtual machines on one host or LUN concurrently. Instead, try to balance your schedule to prevent overusing any resource.
While scheduling prevents slow and degraded performance, testing allows you to know your backup’s recovery readiness. Testing is also important for troubleshooting problems, analysing your tools and deciding whether or not to switch products.
5. Learn How Quiescing and VSS Work
If your VM backups include transactional apps such as database and email servers, you should quiesce them to make sure that they are in a proper state for execution. This backup type is referred to as application-consistent.
Before the backup starts, apps are paused to ensure that any outstanding transactions and writes are written to disk. This step ensures that the server is okay and that no data will be lost if VM recovery is needed. Quiescing only works with those apps that support the pausing and writing of pending data whenever necessary.
VMware Tools has a driver that can work with Microsoft’s Volume Shadow Copy Service (VSS) to quiesce apps before backing them up. Necessity made other vendors come up with a similar driver to serve different operating systems. Ergo, always ensure that you use a supported driver and that your VSS service is enabled and configured to perform application-consistent backups.
6. Snapshots are Not Backups
Virtual machine snapshots, while helpful, should never be used as your primary backup means. Snapshots are okay for short-term backups of virtual machines, but know that you incur penalties whenever you use them.
You see, once a snapshot is created, all the VM’s disk writes are deflected to another new delta disk file making the original disk read-only. As data is written on the delta disk file, it grows in 16 MB increments, with each increment causing a lock on the LUN where it resides, which can degrade performance. Ergo, the more snapshots you run, the more you impact the performance.
What’s more, snapshots take up additional disk space as each one can grow up to the original disk’s size. If you run out of space on your data stores, you risk shutting down all your VMs. Moreover, merging snapshot data into its original after deleting it is a heavy I/O operation.
Snapshots create new virtual disks and link back to the original; therefore, some features may be absent. This can lead to a whole new problem of mapping between snapshots and the original disk. You should use snapshots sparingly and have them deleted when you longer need them.
7. Keep in Mind Fault Tolerance Backup Alternatives
Most virtualization backup products with image-level backups use VM snapshots to halt writes to the disk when backups are running. The Fault Tolerance (FT) feature uses two virtual machines (one primary and one secondary), which, though located on separate hosts, share the same virtual disk file.
Presently, the FT feature does not support snapshots. This makes the process of backing up FT-enabled VMs challenging and you will need to look for alternative methods of backup.
One method involves temporarily disabling the FT feature during the backup process to allow snapshots to be taken. Disabling this feature allows you to preserve the second VM. You can automate this process by using PowerShell as well as pre- and post-backup scripts.
Another method involves the cloning of the VM using either the vCenter Converter or vCenter Server to create another copy. Once you back up the new copy, you can delete the clone.
8. Back Up the Host and vCenter Server Configurations
You can easily rebuild a lost host or vCenter but you will lose your configuration information. It is thus advisable to back up the information periodically. With a backup host, you only back up the VMs and not the individual files residing in their management console. Ergo, backing up the configuration information makes it easier to rebuild the host later.
To backup configuration information:
- For ESX hosts: Use esxcfg-info Service Console command. It will output a lot of configuration information into one text file.
- For ESXi hosts: Use vicfg-cfgbackup command (part of the vSphere CLI). It will output configuration information to one text file.
- For vCenter Servers: You should back up the database containing the configuration information unique to the server including clusters, permissions, resource pools, performance data, alarms and much more. If successful, you can later reinstall the vCenter Server, point it to the backup database, and you will be back up and running. Remember to back up the folder containing the Secure Sockets Layer (SSL) certificate located in the data directory. It contains the SSL certificates that vCenter uses to communicate with ESXi and ESX hosts as well as clients.
The Bottom Line
VMware has made attempts to address the backup challenges that are associated with virtualizing servers. The VMware vSphere 5.1, for example, uses VMware vStorage API for Data Protection (VADP) that works with VMware’s vSphere Data Protection (VDP) or other third-party backup tools.
VADP replaces the VMware Consolidated Backup with an efficient agentless backup system that is based on virtual proxies and does not tax storage resources. In fact, it comes equipped with change block tracking to enable the backup of only those data blocks that you have changed since the previous backup. This reduces the workload of backup tools such as VDP.