GOOD NEWS High Availability XiVO Platform

[wardmundy]

The video was for AstriCon developers primarily. Once we have the movie production company up and running, we'll work on HA - The Movie.

 

[wardmundy]

Didn't take long for the FreePBX knives to come out...

https://www.dslreports.com/forum/r30789104-XiVO-a-suitable-replacement-for-FreePBX~start=60#31010188

 

[dicko]

Just Musing . . .

Historically , HA for Asterisk was basically a homebrewed admixture of abstracting the "state" of an asterisk instance and placing that "state" onto a Master/Slave DRBD filesystem and a "heartbeat" recipe to notice the absense of the "other server" and take the step of firing up local mysql,Apache and network services as appropriate. This I know because five or seven years ago I participated in some of those recipes.

As far as I know the FreePBX recipe likely suborned such a recipe, obfuscated it and then made it commercial, we will never know though.

Things have changed, virtualization is now virtually ubiquitous, no longer do you need to limit the Filesystem to Master/Slave (and the inherent "split-brain" problem) And the whole ,shebang can be on the self same FileSystem (think about the advantages . . .)

GlusterFS is intrinsically Master/Master, Corosync betters Heartbeat, So a simple KVM+corosync recipe is free and "better", for an easy GUI look at ProxMox, it does it all for you ;-) .Two machines, a networked filesystem, a self scripted failover for the machine and it's IP, no double licensing of software, (the Asterisk instance is the same, it just moves to another host.) Would that not work for most?

For XIVO , it uses realtime database and Asterisk has for a longtime had res_corosync for MWI and extension states if you have the two machines able to multicast to each other, perhaps that could be added? and then you could have both instances active for even quicker failover. Just Corosync/BGP to do the routing.

(Perhaps better to have a real proxy between the Asteri and the Internet IMHO though)

JM2CWAE but comments welcomed/anticipated

 

[billsimon]

Contrary to what I have read a few times, I don't think there's any one "right" way to do HA. There are alternatives and compromises.

If you have enough infrastructure, such as redundant network components, a VMWare cluster with hot migration between hosts in separate racks, and distributed shared storage, then you have pretty good HA at the infrastructure level and you can build a single Asterisk instance on it and have some peace. For cloud services, Amazon is probably the closest you can get to this. I would consider running the database in RDS rather than locally on the compute instance.

What do the "big guys" do? Cisco -- and now I'm a few years out of date on this -- replicated everything across a cluster of subscriber (service) nodes and the endpoints knew about several of them so that if one disappeared, it would immediately connect to another. If the whole cluster went down, the last option was a simplified VoIP service running on a router with means to connect to 911, etc.

It's best if you can configure media paths to go direct rather than through the PBX so that if the PBX goes away, the calls can still remain up.

For a while I was working on a vision of stateless/dataless service nodes using FusionPBX (based on FreeSWITCH). I set up a simple two-node cluster and the data layer was separate, on an Amazon RDS cluster. DNS records allow devices to find a list of service nodes (SIP proxies) and pick one that is running. This is pretty good redundancy, HA, and load-balancing, but the db layer is a little slow. A MySQL Galera cluster running on all the nodes would be faster, but more difficult to set up.

 

[wardmundy]

With the advent of dirt cheap SIP phones that are HA-aware, much of the redundancy can be accomplished with something like XiVO-HA without the need to worry about the endpoints. They take care of themselves. We have one of the UTP E-62s that was $29 on Amazon last year. Failover works just as well as our $200 Yealinks. Yes, there can be a 90-second outage in the event of a total server failure. We configure voicemail to deliver messages by email and delete when received so you also don't have to worry about whether voicemails are mirrored. For most businesses and certainly for SOHO implementations, these constraints are certainly acceptable. We're using ImpactVPS servers that are over 2,000 miles apart with NeoRouter to tie them together on the same private network. It's about as fault tolerant as most will ever need, and our cost is roughly $2 per month per server, i.e. less than $50 a year for a totally redundant phone system with zero cost for hardware, networking, and utilities. Could it be done better if money were no object? Absolutely. But, factoring in cost, reliability, and fault tolerance, it's near perfect if all of your trunks and phones speak SIP, IAX, and Google Voice. Ask a CEO or CFO if they're willing to trade a 90-second phone outage once in a blue moon for several thousand extra dollars a month in telephony costs and see how many choose the big ticket option. With the exception of call centers, I'd hazard a guess that the answer will be very close to zero.

 

[chris_c_]

So there is no free open source HA now, for running multiple geographically separated server instances of IncrediblePBX on FreePBX ? Just the commercial module from Sangoma?

 

[dicko]

How do you get that? I saw three recipes, each HA and all open source. (Unfortunately all need a modicum of thinking ;-) )

P.S. The Sangoma solution only works on a LAN

 

[chris_c_]

I'll phrase the question better. There's no add-on, besides the commercial Sangoma one, to enable two standalone IncredibePBX-FreePBX cloud virtual servers, connected on a vpn, do automatic failover. Seems only IncrediblePBX-Xivo has this due to additional open source libraries added on to Xivo.

 

[dicko]

Basically, I think you are correct, there are none. I would suggest additionally that DRBD over a VPN is just begging for a catastrophic Split-Brain failure (if not worse)

 

[wardmundy]

Basically, I think you are correct, there are none. I would suggest additionally that DRBD over a VPN is just begging for a catastrophic Split-Brain failure (if not worse)

I'll let you know. We're running it 24/7 now.

 

[chris_c_]

In case anyone reading this thread is ever interested in trying out HA on a which PBX lacks HA, such as Incredible PBX (FreePBX), PIAF 3, or any other ordinary server, here's a well documented step by step to get HA (Pacemaker Corosync DRBD) running on a VLAN/VPN and protecting Apache. Easily add MySQL and Asterisk.
https://www.lisenet.com/2015/active...cemaker-corosync-and-drbd-on-centos-7-part-3/

 

[ou812]

Just Musing . . .

Historically , HA for Asterisk was basically a homebrewed admixture of abstracting the "state" of an asterisk instance and placing that "state" onto a Master/Slave DRBD filesystem and a "heartbeat" recipe to notice the absense of the "other server" and take the step of firing up local mysql,Apache and network services as appropriate. This I know because five or seven years ago I participated in some of those recipes.

As far as I know the FreePBX recipe likely suborned such a recipe, obfuscated it and then made it commercial, we will never know though.

Things have changed, virtualization is now virtually ubiquitous, no longer do you need to limit the Filesystem to Master/Slave (and the inherent "split-brain" problem) And the whole ,shebang can be on the self same FileSystem (think about the advantages . . .)

GlusterFS is intrinsically Master/Master, Corosync betters Heartbeat, So a simple KVM+corosync recipe is free and "better", for an easy GUI look at ProxMox, it does it all for you ;-) .Two machines, a networked filesystem, a self scripted failover for the machine and it's IP, no double licensing of software, (the Asterisk instance is the same, it just moves to another host.) Would that not work for most?

For XIVO , it uses realtime database and Asterisk has for a longtime had res_corosync for MWI and extension states if you have the two machines able to multicast to each other, perhaps that could be added? and then you could have both instances active for even quicker failover. Just Corosync/BGP to do the routing.

(Perhaps better to have a real proxy between the Asteri and the Internet IMHO though)

JM2CWAE but comments welcomed/anticipated

Hate to dig up an old thread but this is what I am looking for, I do all my recent installs using a proxmox server and I have a customer that is requesting a fail over server sitting on the same local lan just in case of a hardware failure, full backups are kept off site, when you talk about doing this in proxmox are you talking about a cluster with ceph which I believe needs 3 nodes or is there another way, any help would be appreciated as I am not the most seasoned network guy my background is telephoney and I have many asterisk install working well.

Thanks,

 

[dicko]

I think ceph is probably overdoing it, glusterfs is easier and more than adequate running on your two big nodes, it would be rash to have a two node cluster, but a third node could easily be a cheap centrino nuc a or 64 bit atom with 1g memory, and a 80g hdd, you just need it for a 'quorum' to avoid split-brain situations which will almost certainly happen sooner or later with two nodes. Put the machines on the glusterfs and make sure the fencing works, add the HA attribute to the machines and it will 'just work'.

 

[ou812]

Thanks, dicko for the pointers Ill give it a go and see how it works out.

Gary.

 

[wardmundy]

@ou812 FYI: $8.99/mo. buys you native Ceph HA at OVH.com with 99.99% SLA with zero in-house hardware expense or maintenance.