After a doing some work recently for a client creating a suitable off-site backup solution, I have learnt a valuable lesson that many clients may find quite interesting, particularly if this was a real life disaster situation.

As we all know, backups are a critical part of every organisations IT strategy, but how many organisations are truly aware of what their recovery point objective really is?

We all sleep easily at night, knowing that our critical systems and data are protected because they are being securely backed up, by whatever means we may have chosen to perform that service.

Let me now ask you, how many of you have actually tested your backup strategy by actually restoring your systems to a completely new environment?

The reason that I ask you this is because, any IT department will be acutely aware of the rapid growth that we have all seen in the amount of data that we are now expected to store, retain and equally importantly, backup.

I had occasion recently to perform a bare metal restore of what in the grand scheme of things was a relatively small clients entire operating environment consisting of a Windows 2008 Small Business Server with a total of 1.3TB of data and an additional Microsoft SQL Server with approx 600Gb of data.

I can hear you all thinking, less than 2TB of data, that’s not horrendous, and I would be the first to agree with you.

Well, this particular client runs a virtualised VMware infrastructure using VEEAM to backup to a QNAP NAS device, which offers them 8TB of low performance, but, cost effective storage space for their backups.

Our proposal was to offer the client, in addition to their nightly VEEAM backups to their QNAP NAS the additional ability to replicate their systems over the Internet using VEEAM Backup and Replication, Enterprise Plus Edition, utilising the built in WAN Accelerator technology to an off-site datacenter. A logical and sensible approach I hear you say. So let’s move on to putting this into practice.

The first part of creating a replicated off-site backup solution is to create a “seed” of the existing systems, and the easiest and fastest way of getting the initial “seed” is to restore the most recent backup to the replication host, which in this case was located in a hosted datacenter.

With VEEAM, this is a relatively painless and simple process, and it was decided that in order to reduce the load on the clients network and Internet bandwidth, a suitable copy of the clients backup would be transported to the off-site datacenter and restored directly to the replication servers. This is where it became apparent that the bare metal restore process, although simple and easy to do, can, even when your data footprint is relatively small, take a considerable amount of time.

 Would you be surprised to learn that the restore of the 1.3TB Windows 2008 Small Business Server took a total of 39.5hrs?

…and then the SQL Server took 9 hours, comparatively a considerably smaller amount of time, but considerable nonetheless.

OK, if this was a disaster recovery situation, the clients systems would have been restored and in the most part operational within 48 hours, which for some clients may be seen as an acceptable risk, but from my experience even the smallest client can become quite frustrated when they lose access to something as innocuous as email for as little as 30 – 60 minutes, so in all honesty, can you really say that a 48 hour recover point objective is acceptable?

Well, let me continue with the job at hand, which just to remind you was to create a replicated environment for our client.

After the successful completion of the “seed” replica, it was a matter of configuring and running the actual VEEAM Replication Job, which after initial “Seed” must first calculate a digest of the installed disk and then a consequent creation of the “fingerprints” for that disk, which will aid the system in the deduplication and compression of the consequent data that will be transmitted across the WAN. Once again, dependent on the size and quantity of disks that the replicated system contains, this part of the process is also somewhat lengthy.

In the case of the Microsoft SQL Server this initial replication process took 19 hours and for the Microsoft SBS 2008 Small Business Server a staggering further 30.5hrs to complete successfully.

Once the initial replicas have been created, the time required to keep them up to date is drastically reduced, dependent on the schedule you wish to apply, which can be continuous, daily, weekly or pretty much any period you chose, although my recommendation would at the very least, be to perform this daily.

Creating and maintaining an off-site replica, although a time consuming process, and one that appears tedious, may however, one day, in the event of a disaster, provide you with the ability to get your systems up and running again in a very short space of time. Once those replica’s are in place and that unexpected disaster occurs it is simply a case of running up those replicas to their latest restore point and your systems will be back up and running in the space of time it take to run up those replica VM’s.

I think the most important point that anyone should take away from this exercise, is that unless you actually test your backups or replicas at regular intervals, you will never know what your TRUE recovery point is. Wouldn’t it be better to know NOW and not have to wait until the day you REALLY need them.