RECOMMENDATIONS Where to store CEL, CDR, and Recordings

[krzykat]

For anyone and @dicko that has played with storing records remotely, what do you think the best strategy is? CEL and Monitor (if you record many calls) obviously are the big disk space hogs. So I'm looking at storing these on a remote disk system. I'm thinking that this will allow me to not only save a lot of disk space and potentially speed up a PBX, but could also be utilized for doing failovers. If I've got a hosted PBX on Vultr for example, I could theoretically have a failover server on Digital Ocean. Then if I kick over to the failover server, they'd still have access to the records. I'm contemplating whether to have the CDR on the remote or not as well and I'm just wondering how others have dealt with this scenario. I think the Mixmon should write to /remote/asterisk/site1/monitor - and if I write the CDR's they'd need to have a different SQL table for each server. If CDR records are kept on the PBX, then we'd still have access to everything even if the remote goes away, but then I'd need to sync the CDR to the failover if that happens.

Anyhow - just looking to open a discussion to see what others thing that have contemplated these scenarios.

 

[dicko]

Two different things here, the cel table in asteriskcdrdb does grow quite quickly but if carefully purged is not going to be too big, first question is "is it useful to you" . if it is not, disable it.

You can easily change where your cdr's go to in the FreePBX advanced settings. but you have to think "is it a good idea?" each "write" needs to open a new connection to the remote mysql server, not a biggy for cdr's but a little more impactful for cel records. the remote server needs to be accessible and if serving more than one backend you need to add something unique to the record. You can also define as many alternate cdr destinations by hand using odbc you might want to look at pgsql or even mondodb.

There is another problem with mysql tables as used in FreePBX in that that if they grow too large, you can purge the old records, but the table doesn't actually get smaller. So you need a careful scheme to purge no longer useful records to not run into problems.

Offloading monitor files is not too big a problem but it depends on where you are sending them, nfs shares go "stale" for any number of reasons, CIFS just sucks, sshfs mounts are one possibilty but you need to cache the calls locally before comitting them to remote storage

https://community.freepbx.org/t/change-call-recordings-path/25082


I suspect you are attempting a HA solution, which while possibly feasible would not be my choice of direction, I like rsnapshot to a remote location for /var/spool/asterisk/ which covers both monitor files and voicemails which with a little care can be sourced from several instances.

Good luck, it won't be a walk in the park though ;-)

 

[krzykat]

Two different things here, the cel table in asteriskcdrdb does grow quite quickly but if carefully purged is not going to be too big, first question is "is it useful to you" . if it is not, disable it.

You can easily change where your cdr's go to in the FreePBX advanced settings. but you have to think "is it a good idea?" each "write" needs to open a new connection to the remote mysql server, not a biggy for cdr's but a little more impactful for cel records. the remote server needs to be accessible and if serving more than one backend you need to add something unique to the record. You can also define as many alternate cdr destinations by hand using odbc you might want to look at pgsql or even mondodb.

There is another problem with mysql tables as used in FreePBX in that that if they grow too large, you can purge the old records, but the table doesn't actually get smaller. So you need a careful scheme to purge no longer useful records to not run into problems.

Offloading monitor files is not too big a problem but it depends on where you are sending them, nfs shares go "stale" for any number of reasons, CIFS just sucks, sshfs mounts are one possibilty but you need to cache the calls locally before comitting them to remote storage

https://community.freepbx.org/t/change-call-recordings-path/25082


I suspect you are attempting a HA solution, which while possibly feasible would not be my choice of direction, I like rsnapshot to a remote location for /var/spool/asterisk/ which covers both monitor files and voicemails which with a little care can be sourced from several instances.

Good luck, it won't be a walk in the park though ;-)

I do find CEL useful, but I want to pair it down to be more readable, and I think I can do that after the fact I think there is too much stuff in there that most people don't care about, but certainly, who answered, was the call parked, who grabbed it next, etc. I find useful, so I plan to "Tweak" it to give a more user friendly feel. I can probably also use the data to add what I think is a missing disposition to CDR's ... which is "Answered Elsewhere" - happens all the time when using ring groups and makes looking at stuff a little more complicated for the users.

My idea is that if we store the records remotely - we can accomplish a couple things at the same time. As you eluded to, it sets you up for redundancy in a HA situation, but it also allows you to keep much more data than you would otherwise. On a 25Gb Hosted VM, you can eat that up pretty quickly with CEL's and Mixmon, but if you put it elsewhere, such as on a FreeNAS or Synology, then your diskspace becomes so cheap, that you could almost keep things forever. I'm thinking CDR's for 7 years, CEL and Mixmon for 3 years.

I'm planning on using SSHFS for mounts, and MariaDB for the SQL. (These parts currently work in testing).

I like the caching concept, but don't think i have it working quite right yet. Will it allow me to cache CDR, CEL, and Mixmon ??

The HA solution I'm looking at is closer to what you propose. My thought is that I have one server on Let's say Vultr, and another on the something like Hosting73. Daily I'd use a modified version of Wards backup / restore so that the failover is pretty current, and then of course if it kicks over, the CDR, CEL, and Mixmon would be consistent with the sshfs mount. This strategy would also open up new avenues to have user backend that would allow reporting, and histories without ever touching or exposing the PBX.

Overall, I think its a very plausible solution, and the only issue would be to make sure that you have good backups of the sshfs mount.

Comments welcomed.

 

[dicko]

If you want HA, IMHO opinion you will need a solid network based block-storage one node of which is "local" to the PBX which of course needs to be as big as what is needed to be 'current; , an easy one is glusterfs a more robust one is ceph , (drbd not so good),either one really needs 3 nodes for reliability, if you have that then you can commit everything transient to it, The mysql records are written individually so low bandwidth and just need a solid network to complete, the monitor files are streamed live, so in your interest they MUST be cached locally before being committed to the final destination.

It works pretty well with corosync mediating but personally I have moved on to a robust clustered SIP proxy and cloud based PBX's , the good providers API's let you dynamically fire up and down good and bad servers (and repair them in the background) , truth is it a few minutes of downtime when it happens, again pragmatically it has never happened to me yet.

 

[krzykat]

If you want HA, IMHO opinion you will need a solid network based block-storage one node of which is "local" to the PBX , an easy one is glusterfs a more robust one is ceph , (drbd not so good),either one really needs 3 nodes for reliability, if you have that then you can commit everything transient to it, The mysql records are written individually so low bandwidth and just need a solid network to complete, the monitor files are streamed live, so in your interest they MUST be cached locally before being committed to the final destination.

It works pretty well with corosync mediating but personally I have moved on to a robust clustered SIP proxy and cloud based PBX's , the good providers API's let you dynamically fire up and down good and bad servers (and repair them in the background) , truth is it a few minutes of downtime when it happens, again pragmatically it has never happened to me yet.

Yes, I'm with you on the cloud based PBX, providers, and API's. What are you using for clustered SIP Proxy - It's been years since I've looked at OpenSER, you using OpenSIPS or ?

 

[dicko]

Well it was naked Kamailio, much easier to wrap that up in dSipRouter, SRV records on your DNS to direct the traffic and a bit of background syncing with https://kamailio.org/docs/modules/4.1.x/modules/db_cluster.html , yep thats a RTFM ;-). But If you can arrange for BGP it makes it a easier

 

[krzykat]

I know how and am just fine with reading ... Just nice to know which book to crack - otherwise you can read yourself in circles. Much appreciate the pointers and school of hard knocks you've given.