FOOD FOR THOUGHT Failover Methodology - High Availability

[krzykat]

What in your opinion is the best failover methodology with PIAF?

Here's what I've come up with. Server on Wable, another server on RentPBX. They're both setup identical - with freepbx backup /restore from master to backup server. Once a day seems fine for my purposes, but if you wanted more frequent, I guess you could. Then I signed up for an account with DNS Made Easy with failover. I set the primary and secondary with a TTL of 60 (do I need to do that?) - and then set the failover checking to HIGH (should find it within 2 minutes I believe). Then on the sip device, set the DNS to fixed of DNSmadeeasy primary and secondary DNS. And of course set the SIP device to register every 60 seconds. I think this should give you a semi high-availability setup.

The other option that I was considering is to allow the SIP device to have a SIP server 1, and SIP server 2. But I rather think my first listed methodology is cleaner giving one DNS name such as mypbxserver.mydomain.com

Agree / Disagree ??

 

[wardmundy]

Haven't played with this in a long time, but my recollection was that the easy way to handle failover was to use IPtables to block trunk registrations on the secondary server until the first server failed. Then unblock the ports, change DNS, and you're back in business.

 

[voip_user]

Hmm the best method of failover, I have only played with DRBD, with heartbeat, and I have only done it on straight asterisk. There are a few more out there, but I think the best bet if your not familiar with making your own is using a 3rd party solution like HAAST. They give a free license to try it out.

 

[wardmundy]

@krzykat: Take a look at this old thread. Lots of good ideas to play with.

See also: http://pbxinaflash.com/community/in...-for-big-deployments.16115/page-2#post-115892

 

[marlur6]

Then I signed up for an account with DNS Made Easy with failover.

I am trying to set up failover with two PIAF servers also. I signed up for an account with DNS Made Easy with failover also. How did you setup the monitoring part? For me it works fine if I set protocol to HTTP port 80. I think it would be better to monitor UDP on port 5060. However that requires a UDP send string and timeout. I could not figure out what to set that to. Anybody know what should be in UDP send string. What is the best protocol and port to monitor? Thanks.

 

[krzykat]

I've done the same as you with the port 80. I too agree that you really should do the UDP 5060 check, but I haven't gotten to the point of knowing what to call and expect reply for yet. If you get it let me know. You have DNS Made Easy IP's in your whitelist, so they can easily get through, right? I got the list if you need.

 

[marlur6]

I've done the same as you with the port 80. I too agree that you really should do the UDP 5060 check, but I haven't gotten to the point of knowing what to call and expect reply for yet. If you get it let me know. You have DNS Made Easy IP's in your whitelist, so they can easily get through, right? I got the list if you need.

I guess we are in the same boat. Yes I got their IPs off their DNS failover tutorial ( http://help.dnsmadeeasy.com/dns-failover/configure-dns-failover/configure-dns-failover/ ) and put them in iptables. I started a thread about it over on DSL reports but got no answer yet. http://www.dslreports.com/forum/r30446473-PBX-UDP-send-string-for-DNS-failover

 

[marlur6]

The other thing I am trying to setup is IMAP storage for voicemail. The reason is if a voicemail is left on the backup server then the main server gets back online you will not be able to get the voicemail on the backup server from the phones. I started a thread about that on DSL to. http://www.dslreports.com/forum/r30446438-Asterisk-Gmail-IMAP-storage-for-PIAF I need to figure out how to recompile Asterisk with IMAP support.

 

[krzykat]

I have very few clients that even want the voicemail left on the server. Most everybody now likes and uses the e-mail methodology. We take the e-mail, convert it to MP3 so that they can even listen to it on their mobile devices directly and e-mail it to them with the delete after e-mailing option engaged. Failover should be used as an emergency in my opinion, and if the only inconvenience is your voicemail not syncing across .... beats the hell out of being down. Maybe have your secondary failover server set to automatically e-mail voicemails across so that they would never be lost or missed if a message got left there, and then the primary came back online.

 

[marlur6]

I use VM to email myself. The problem is some of my users don't use email or they check it infrequently. For myself I like to let VM on the server after emailing so I get message waiting indicator on my phones. If I can get it all synchronized to my email it would be awesome. If all my users had email I probably would just email them from the backup server like you suggested.

Hopefully someone can give us some info about UDP send string.

 

[krzykat]

Are you doing backup/restore from primary to secondary daily, every 12 hours or what is your criteria?

 

[marlur6]

I am still setting it up yet. I don't have backup restore automated. Not sure what I am doing for frequency yet. I only have my own extensions on my new failover domain. I want to get all the bugs worked out before I put all my users on it.

 

[krzykat]

In my mind, there are a couple of different ways to do the backup/restore. One is that you give the servers access to each other and then the backup is actually done across primary to secondary. Then the secondary does a restore to keep it current. Another option would be to use a 3rd piece - let's call it backup FTP - where you could write your backups to and then the secondary grabs them from and does its restore to keep itself current. I use the first method, as I couldn't see enough good reason for the second. Your setups can determine how much or little you need or want to copy across. CDR's can be large, but maybe you even use a 3rd CDR server that handles that part (I haven't done this, but have thought about it).

You can use the backup/restore module in FreePBX (which is what I've been doing), or also - Ward has something that I haven't gotten to investigate yet: http://nerdvittles.dreamhosters.com/pbxinaflash/1755/docs/dailybackup.pdf

 

[Michelle Dupuis]

Some food for thought...what kind of an 'outage' will your HA cluster respond to? An outright failure of a server is easy, but what if Asterisk is alive but not bridging calls? What if the link to your data center goes down (but Asterisk is still alive)? What if the hard disk becomes flaky (corrupting voicemails, etc) but Asterisk is still alive? I would suggest you focus on the DETECTION part first.

Next, if you share any type of storage (DRBD, NFS, etc) what if a peer feels and begins corrupting your disk? Have you just corrupted the data used by the peer - so it won't run properly either?

You're also starting to touch on the issue of sharing security info between peers. Some HA software integrates with security software to move all security info to the active peer (eg: banned' IP's), or talk to an external firewall for banning IP's so your cluster doesn't have to be aware of security control.

If you want to design your own HA solution have a look at:

http://www.voip-info.org/wiki/view/Asterisk+High+Availability+Design

it covers a lot of the issues to consider. Essential if you want to build your own, OR, buy something.