SUGGESTIONS Disaster Recovery

[Brianmac]

I'm a complete VOIP newbie, but have been dealing with smoothwall routers for years. So I am comfortable in linux, putty, iptables, winscp...etc and I am a network consultant(windows),so I have a pretty knowledgeable background.

I love this stuff, but one questions....

How do you deal with Disaster Recovery. I have always dealt with servers, by having a duplicate system, and using imaging. This way I can have a system back up in 30 minutes. How do you deal with PIAF, in case the PC/server dies? I have been reading the forums, and one response said that he had many servers in place and had a blank server(with current updates) and if a PiaF fails, he can load the configs and have that office up in 10 minutes. Is there any docs on something like this?

I have a commercial customer that has a small NEW business that I would like to test his with, but very unfamiliar with disaster recovery on this stuff.


Thanks for any information...

Brian

 

[atsak]

I use hyper V (you could use VMWare or Virtualbox as well) which replicates things everywhere automatically in near real time. In testing the failover process takes less than a minute.

There's other ways like using imaging as you suggest, automated backups using FreePBX are pretty good too.

 

[Robert-BCC]

The GUI has a backup / restore function under the admin menu. Even including system recordings and voice mail for a 10+ person office, it's about 30MB. Recovery requires that you have the same version in place to ensure that things like mysql entries can be restored properly.

For the small office I'm minding, their PiaF is on a hosted VM off-site. So I'm considering putting their current config onto a CUBOX and then installing that puppy on site. After the failure, all of the phones would still try to register to the off-site PiaF. So part of the recovery procedure would be to change the DNS entry so that all of the phones use the new server. Given the current one hour TTL on the A record, this isn't as slick as the 10 minute recovery you described, but I can't think of a better way to do it.

The other thing you should consider is automatic failover from your SIP provider. If a call can't be routed to your PiaF, you can set it up so someone's cell phone would automatically take the call. Here's how that looks on the Vitelity web site:

 

[wardmundy]

In terms of speed of recovery from a disaster, nothing beats a virtual machine environment where you can take snapshots of your system and recover in minutes. The downside is lack of support for POTS landlines without using external SIP adapters such as the OBi110.

For normal dedicated hardware, keep in mind that system changes typically are pretty rare except when you're setting up the server. Once it's stable, you can shut down the server and make an image backup with something like Clonezilla. Takes a bit longer to restore but still works great.

The other option is to build two identical Incredible PBX 11 servers with Asterisk 11 and FreePBX 2.11. Then use the included incrediblebackup script to make a regular backup which you can copy to a flash drive and keep off site. When disaster strikes, boot up the other server and run incrediblerestore with your backup image. Shouldn't take more than 5 minutes to be back in operation. Incredible Backup 11 also works well with two PIAF-Green servers configured with Asterisk 11 and FreePBX 2.11. If you want to mix and match server platforms, there's a whole thread explaining how to do it. Good luck!

 

[Brianmac]

Thanks for all the responses, it was very helpful.

I see a lot of suggestions on using a virtual machine... I still don't get it. I know for backups and such, its GREAT. My issue is, lets say my system's motherboard dies. Unless I have another machine, I don't see how VM is really going to help me. I have always suggested to clients to buy duplicate servers with EXACT same hardware. I always split the drives, Cdrive= for operating system and apps and Ddrive for the data. Then the 2nd machine, has a different OS and syncs the Ddrive from the production box every hour. I then have the production box image itself once a week that is copied to backup server. If the production server dies, then I just take the backup server, apply the image of the production server to it, boot and off I go. worst case I lose an hour of data on a worst case scenario. Note on the servers I am using raid,also.

I read in the forums(see below) about having a ghost machine offsite and then applying the config to it. To me this is VERY interesting, but I haven't been able to find much more data on it.Then I could standardize on a server and have a spare at my location. I only use Poweredge servers with DRAC - helps with being able to reboot and see whats going on if the OS isn't booting up....

Thanks,
Brian

http://pbxinaflash.com/community/in...ery-virtual-machine-and-other-questions.3676/
OTA, response# 14

I have 1 warm spare PiaF system in the rack. Default blank config but kept up-to-date with modules & such. If one of my clients' PBXs were to die, I'd just upload the correct config to it, change my DNS server settings and they're up & running in under 10 minutes. And the best part -- I don't even have to set foot on-site. If I was really desperate, I could pull the whole thing off using MidSSH on my Blackberry. Wouldn't be fun, but it is possible. Show me ANY commercial PBX from the big boys that can do that.

 

[Robert-BCC]

Worth noting that for the quoted solution to work the TTL on the A record pointing to the PBX must be less than 600, and the handsets have to respect the TTL.

 

[GomezAddams]

Thanks for all the responses, it was very helpful.

I see a lot of suggestions on using a virtual machine... I still don't get it. I know for backups and such, its GREAT. My issue is, lets say my system's motherboard dies. Unless I have another machine, I don't see how VM is really going to help me. I have always suggested to clients to buy duplicate servers with EXACT same hardware. I always split the drives, Cdrive= for operating system and apps and Ddrive for the data. Then the 2nd machine, has a different OS and syncs the Ddrive from the production box every hour. I then have the production box image itself once a week that is copied to backup server. If the production server dies, then I just take the backup server, apply the image of the production server to it, boot and off I go. worst case I lose an hour of data on a worst case scenario. Note on the servers I am using raid,also.

I read in the forums(see below) about having a ghost machine offsite and then applying the config to it. To me this is VERY interesting, but I haven't been able to find much more data on it.Then I could standardize on a server and have a spare at my location. I only use Poweredge servers with DRAC - helps with being able to reboot and see whats going on if the OS isn't booting up....

Thanks,
Brian

http://pbxinaflash.com/community/in...ery-virtual-machine-and-other-questions.3676/
OTA, response# 14

Virtualization gives you all the benefits and capabilities you describe above, and the added benefit that you don't need the same hardware - you can recover to *anything* (so long as it has enough RAM and CPU). You also don't need imaging software. Your OS disk is just a file, and there are host utilities for creating copies of it automatically.

You don't need DRAC, or iLO, or whatever. You get the equivalent of a network based KVM for free.

Trust me - throw ESXi on something and install a few things as virtuals - you'll never go back!

 

[TheMole]

Virtualization gives you all the benefits and capabilities you describe above, and the added benefit that you don't need the same hardware - you can recover to *anything* (so long as it has enough RAM and CPU). You also don't need imaging software. Your OS disk is just a file, and there are host utilities for creating copies of it automatically.

You don't need DRAC, or iLO, or whatever. You get the equivalent of a network based KVM for free.

Trust me - throw ESXi on something and install a few things as virtuals - you'll never go back!

agreed, once you try it you will never go back. ESXi is free and now supports more memory with the free license in 5.5

there is a great blog and forum over at www.thehomeserverblog.com (which looks to be down right now,but has a great amount of info to get you going)

you can automate your ESXi backups with the ghettoVCB script (http://www.virtuallyghetto.com/2013/01/new-updates-for-ghettovcb-ghettovcb.html).

i have 7 VMs (with room to spare) running on an Intel Core i7 4930K 64GB RAM. one is a PIAF install. no problems what-so-ever. all the VMs backup to a NAS nightly and copies are kept for 8 days.

 

[GomezAddams]

Just for grins on a rainy day a while back, I installed ESXi 4.1 on an old Pentium D (dual core CPU) with 4GB of ram and brought up a PiaF, Server 2003, and XP virtuals. They actually performed about as well as if they were running native on the same hardware. They were certainly usable.

Oh, and Veem has some interesting free backup utilities.