FYI FreePBX 15 Restore Module

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
The original problem was a COMMENTS field in the OSS Endpoint Manager tables. That module is not yet (and maybe not ever) going to work with FreePBX 15. That was Andrew's baby, and he quit supporting it 5 years ago. He's now left Sangoma as well. Once that problem was resolved, a couple more problems popped up. If you have an Incredible PBX 16-15 server you don't mind destroying, you can redownload and reinstall the backup16-15 script above. That will load a patched backup module and let you see what a fine job the restore does on most of the FreePBX 13 modules... until it blows up because of a mismatch of the admin manager credentials, problems with legacy CDR data, and an EOL error with the npm stuff needed by UCP. By the way, when you first choose the Backup & Restore option in the GUI, that may blow up, too. But you can reload the GUI, and it comes up properly after that. Let's call that one "a feature." Fun times!
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
OK. I think we're ready for the pioneers.

:party::party::party:

You'll need to have an Incredible PBX 13-13.10 server that you wish to restore to a NEW Incredible PBX 16-15.1 server.

On the Incredible PBX 13-13.10 server:
1. Load the latest backup module for FreePBX 13: /root/gpl-install-fpbx backup
2. From the GUI, make a new backup.
3. Copy the backup from /var/spool/asterisk/backup to your desktop PC.

Create a NEW Incredible PBX 16-15.1 server with CentOS 7 using the tutorial: http://nerdvittles.com/?p=30236

On the Incredible PBX 16-15.1 server:
1. Login to the GUI as admin with the password you set with admin-pw-change.
2. Quirk #1: The first time you access Admin->Backup, it will blow up. Hit your browser's Back button twice to return to Dashboard.
3. Access Admin->Backup & Restore again and click on the Restore tab.
4. Drag your 13-13 backup file to the designated area on the form.
5. Once the backup is loaded, click RunRestore button to begin.
6. When the whirring stops, there will be an error message. Ignore it. Don't click anything yet.
7. Drop down to the Linux CLI and login as root.
8. Run: /root/restore-fix
9. Go back into the GUI with your browser, close the Restore dialog, and return to the Dashboard. Ignore the warning about Bind Ports.
10. Click Settings->SIP Settings->SIP Settings (pj_sip) and scroll down to UDP. Click YES then Submit then Apply Config.
11. Return to Dashboard and then verify that all your Incredible PBX 13-13.10 data was ported over correctly.

Please report glitches. :cool:
 

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
@wardmundy it mostly worked! Here are a couple of observations and issues found so far:

1. It loads the old version of IPCHECKER into the root directory. (The one where you have to define account#1, etc. at the top.) Its not like the one in IncrediblePBX-13-13.10
2. The caller ID for extensions is mucked up a bit. For what should be "Name"<8883219876>, it loads as /"Name/"<8883219876> . The slashes should not be there.
3. Voicemail settings did not move across. Voicemail is set to OFF on each restored extension.
4. Apparently /etc/asterisk/extensions_custom.conf settings do not get moved over. My misc applications or anything added in that file are not restored.
5. The callerid superfecta default lookup scheme is toggled to off.
6. I had to restart with a fwconsole restart because commands such as "sip show peers" did not work. Sip was not recognized as a command until after the restart for some reason.
7. User manager users did not migrate. The screen is empty.
8. Applications--> Wake up calls - On the settings tab, if you make a change and click SUBMIT, the screen aborts. A refresh doesn't fix it.
9. Reports --> Asterisk Info - Screen bombs
10. Voicemail files in /var/spool/asterisk/voicemail did not get restored. They were backed up on the 13-13.10 backup file.
11. CDR did not migrate. It was in the 13-13.10 backup.
12. Admin-->Backup does not work. Following error returns:
Code:
Running with: /usr/sbin/fwconsole backup --backup='9da86930-594f-420d-94a5-1ad9c634213c' --transaction='38bef02d-7270-43bc-a552-c8eb848b5ace' >> /var/log/asterisk/backup_38bef02d-7270-43bc-a552-c8eb848b5ace_out.log 2> /var/log/asterisk/backup_38bef02d-7270-43bc-a552-c8eb848b5ace_err.log & echo $!
Running Backup ID: 9da86930-594f-420d-94a5-1ad9c634213c
Transaction: 38bef02d-7270-43bc-a552-c8eb848b5ace
Starting backup Full_Backup_(Monthly)
This backup will be stored locally and is subject to maintenance settings
Storage Location: /var/spool/asterisk/backup/Full_Backup_(Monthly)/20190728-093732-1564321052-15.0.16-1835835954.tar.gz

In Multiple.php line 151:
                                        
  Invalid argument supplied for foreach()
                                        

backup [--backup BACKUP] [--externbackup EXTERNBACKUP] [--dumpextern DUMPEXTERN] [--transaction TRANSACTION] [--list] [--warmspare] [--implemented] [--filestore FILESTORE] [--restore RESTORE] [--modules MODULES] [--restoresingle RESTORESINGLE] [--backupsingle BACKUPSINGLE] [--singlesaveto SINGLESAVETO] [--b64import B64IMPORT] [--fallback]
13. Update 733 for skinny port 2000 is not applied.
14. /var/spool/mqueue has wrong permissions. Throws error on dashboard for mail queue status.

That's all I've found so far.
 
Last edited:
  • Wow
Reactions: wardmundy

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
@kenn10: Gradually working through your list here...

1. Doesn't look like /root gets restored at all. The ipchecker mismatch was because I had the wrong one in 16-15. Fixed.
2. I'll add this to the restore-fix script.
3. See #4 below concerning voicemail settings and content.
4. Default backup settings don't pick up everything you would want. These have to be added on the 13-13 backup side. Here's what I used:


5. Looks like you manually have to turn Superfecta back on until I can find the setting: Admin -> CID Superfecta -> Action -> toggle ON
6. fwconsole restart is already in the restore-fix script.
7. Looks like you have to rebuild User accounts in User Manager by selecting User tab and then PBX Internal Directory. Then you can add back user accounts.
8. Hotel Wakeup Calls needed an update from the Edge repo which you can enable in Advanced Settings once you switch to FreePBX repos. I've added it to newbackup16-15.tar.gz.
9. Looks like this uses Asterisk's internal web server which is probably not activated. Will dig deeper. See https://thetechnologygeek.org/install-asterisk-web-based-gui/
10. /var/spool/asterisk/voicemail directory should be added to your default 13-13 backup template. I may have forgotten it above.
11. Nerd Vittles RSS Feed also wouldn't come up after the restore. This has been added to restore-fix script.
12. Making a FreePBX Backup on 16-15 seems to blow up. Stick with /root/incrediblebackup16 for the time-being. Or take a (free) snapshot on Vultr.
 
Last edited:

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
Based on your info above, I made a new backup and restored it. It worked better this time. That fixed the Asterisk Info report.

I discovered that in the 13-13.10 Backup setup, if you don't put a * after the directories for MOH, t*f*t*p, etc., the files aren't copied.

Also, the Admin-->Custom Destinations do not import, even though the extensions_custom.conf file is now copied over. I haven't played with it to see if the order of backing up files matters. I currently have the extensions_custom.conf file as the first entry in the backup routine.
 
Last edited:
  • Like
Reactions: wardmundy

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
I've made a half dozen updates to newbackup16-15.tar.gz and (included) restore-fix script since your last post. Might wanna redownload and then run updated install-backup script again. :)
 

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
I've made a half dozen updates to newbackup16-15.tar.gz and (included) restore-fix script since your last post. Might wanna redownload and then run updated install-backup script again. :)
Greatly improved. Issues I now see:

1) The emergency CID on the extension still has the slashes around the name. Probably need to check route CID's too.
2) Still had to do a manual stop and start (fwconsole stop, fwconsole start) for Asterisk CLI to accept a command such as sip show peers. I saw the restart during the restore process and fix-restore, but I still had to do it again to get the commands to work in the CLI.
3) Custom destinations did not populate.
4) Misc applications set to Enable=No (perhaps because the associated custom destination was not there.)
5) Still no CDR data moved over.
6) Fresh backup on 15-16.1 still aborts.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
Still to do:

1. First access of Backup & Restore bombs. Second access works fine.
Code:
*var*www*html*admin*libraries*BMP*Database.class.php

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 1000 bytes
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
@kenn10: Your latest list...

1. Fixed.
2. Added another stop and start to install-backup script.
3 & 4. Looks like miscapps table and CustomApps in kvstore table and miscapps in featurecodes table don't get populated during the restore. So those would have to manually be added using the GUI options for Custom Destinations and Misc Applications on both platforms. The restore app isn't something most people would use regularly. Probably only once to migrate to 16-15. So knowing where the land mines are so they could be cleaned up will be very helpful. Simple cut-and-paste here should do the trick for these two components.
5. Perhaps a MySQL option in 13-Backup will add option to copy over the asteriskcdrdb database and tables. cel table structure is identical in 13 and 15. cdr table in 15 adds 3 additional fields:
Code:
linkedid, varchar32 
peeraccount ,varchar80
sequence,int11,default= 0
6. BUG. Stick with incrediblebackup16 in the Linux CLI for now.
 
Last edited:
  • Like
Reactions: kenn10

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
@kenn10: Your latest list...

1. Fixed.
2. Added another stop and start to install-backup script.
3 & 4. Looks like miscapps table and CustomApps in kvstore table and miscapps in featurecodes table don't get populated during the restore. So those would have to manually be added using the GUI options for Custom Destinations and Misc Applications on both platforms. The restore app isn't something most people would use regularly. Probably only once to migrate to 16-15. So knowing where the land mines are so they could be cleaned up will be very helpful. Simple cut-and-paste here should do the trick for these two components.
5. Perhaps a MySQL option in 13-Backup will add option to copy over the asteriskcdrdb database and tables. cel table structure is identical in 13 and 15. cdr table in 15 adds 3 additional fields:
Code:
linkedid, varchar32
peeraccount ,varchar80
sequence,int11,default= 0
6. BUG. Stick with incrediblebackup16 in the Linux CLI for now.
Well, I've taken the test drive and done what I can do. Now if others will hop in and give it a try, perhaps more bugs will be exposed. If we're lucky, there won't be much more to find and folks can start wholesale system upgrades to the Incredible16-15.1 release!
 

sortons

Member
Joined
Aug 9, 2018
Messages
39
Reaction score
6
Clean install of 16-15.1 - followed the instructions in posts #44 & 42 (in that order).
The restore encountered the error as described above. After ./restore-fix and UDP enable the dashboard came up alright. Trunks, routes, extensions all in place. I pointed an IP phone to the 16-15.1 IPBX and was able to place a call and receive a call through the 13-13.10 configured trunk / routes.

CDRs and voicemails did not come over (I may have missed those in the 13-13.10 backup.)
 

sortons

Member
Joined
Aug 9, 2018
Messages
39
Reaction score
6
...continued..
.
Reports -> Asterisk Info comes up with:
Whoops\Exception\ErrorException thrown with message "file_get_contents(http://[email protected]:8088/ari/endpoints): failed to open stream: Connection refused"

All trunks connected properly to the providers - only tested one as described above (the test 13-13.10 IPBX was shutdown during the 16-15.1 testing.)

Very promising, indeed - thank you!
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
Importing CDR and CEL data from Incredible PBX 13-13.10

Well, we're making progress. We have successfully imported all the CDR and CEL data from a couple of our 13-13 servers. Here's what we are up against. With the new FreePBX 15, they have changed BOTH the CDR table structure and moved to an InnoDB database. These are MAJOR changes and explains why they haven't tackled the import issue. We have sorted it out, we think.We have proven that the old stuff displays in the CDR reports on 16-15 and, more importantly, new calls get logged properly. The hurdles were/are formidable. First, MySQL won't let you import data unless the old and new table structures match. That means removing the new fields temporarily, doing the import, and then adding the new fields back in so that FreePBX 15 CDR functions still work. I have no idea what the 3 new fields do, but you probably need them with FreePBX 15 so they have been put back after the 13-13 CDR restore. You can review the import-cdr1313 script to see how hairy this actually gets.

Before you can import your CDR and CEL data from Incredible PBX 13-13, you first have to create it. This will NOT damage or modify your 13-13 CDR/CEL data! It's merely a snapshot.

UPDATE: Existing CDR/CEL data on 16-15 platform gets preserved and restored as part of running the latest script.


1. On the Incredible PBX 13-13 platform, issue commands:
mysqldump -u root -ppassw0rd --single-transaction --quick --lock-tables=false asteriskcdrdb > asteriskcdr1313.sql
gzip asteriskcdr1313.sql
/root/add-ip myserver1615 server1615-ip-address
2. On the Incredible PBX 16-15 platform, grab the 1313 CDR snapshot:
scp [email protected]:/root/asteriskcdr1313.sql.gz /root/.
3. Then run /root/import-cdr1313

To get the import-cdr1313 script, the easiest way is to redownload and untar newbackup16-15.tar.gz:
Code:
cd /root
wget http://incrediblepbx.com/newbackup16-15.tar.gz
tar zxvf newbackup16-15.tar.gz
rm -f newbackup16-15.tar.gz
 
Last edited:
  • Like
Reactions: sortons

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
Importing CDR and CEL data from Incredible PBX 13-13.10

Well, we're making progress. We have successfully imported all the CDR and CEL data from a couple of our 13-13 servers. Here's what we are up against. With the new FreePBX 15, they have changed BOTH the CDR table structure and moved to an InnoDB database. These are MAJOR changes and explains why they haven't tackled the import issue. We have sorted it out, we think.We have proven that the old stuff displays in the CDR reports on 16-15 and, more importantly, new calls get logged properly. The hurdles were/are formidable. First, MySQL won't let you import data unless the old and new table structures match. That means removing the new fields temporarily, doing the import, and then adding the new fields back in so that FreePBX 15 CDR functions still work. I have no idea what the 3 new fields do, but you probably need them with FreePBX 15 so they have been put back after the 13-13 CDR restore. You can review the import-cdr1313 script to see how hairy this actually gets.

Before you can import your CDR and CEL data from Incredible PBX 13-13, you first have to create it. This will NOT damage or modify your 13-13 CDR/CEL data! It's merely a snapshot.
On the 16-15 side, the CDR and CEL tables get their data purged as part of the import so don't use this IF you have existing CDR/CEL data you wish to preserve.

1. On the Incredible PBX 13-13 platform, issue commands:
mysqldump -u root -ppassw0rd --single-transaction --quick --lock-tables=false asteriskcdrdb > asteriskcdr1313.sql
gzip asteriskcdr1313.sql
/root/add-ip myserver1615 server1615-ip-address
2. On the Incredible PBX 16-15 platform, grab the 1313 CDR snapshot:
scp [email protected]:/root/asteriskcdr1313.sql.gz /root/.
3. Then run /root/import-cdr1313

To get the import-cdr1313 script, the easiest way is to redownload and untar newbackup16-15.tar.gz:
Code:
cd /root
wget http://incrediblepbx.com/newbackup16-15.tar.gz
tar zxvf newbackup16-15.tar.gz
rm -f newbackup16-15.tar.gz

Hmmm. No worky for me. It wiped out the CDR I had generated on the 16-15.1 system but did not load in the 13-13 data. The script did generate two SQL command files but the data did not get loaded into the database.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
Any errors? Do you need a fresh asteriskcdrdb database with the CDR and CEL tables??

Perhaps I need to include an empty asteriskcdrdb.sql just to be sure we start from the right place.
 

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
Any errors? Do you need a fresh asteriskcdrdb database with the CDR and CEL tables??

Perhaps I need to include an empty asteriskcdrdb.sql just to be sure we start from the right place.
Looking at the script, I don't see where you actually import the cdr1313.txt and cel1313.txt into the database? The files are created and at the end erased.
 

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
Running the script line by line, this error pops up:
Code:
 mysql -u root -ppassw0rd asteriskcdrdb < asteriskcdr1615.sql
ERROR 1136 (21S01) at line 65: Column count doesn't match value count at row 1
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
Download the newbackup16-15.tar.gz file again, and it has the asteriskcdrdb template in it. Then just restore it before running the updated script:

mysql -u root -ppassw0rd asteriskcdrdb < asteriskcdrdb.sql
 

kenn10

A lesser geek
Joined
Dec 16, 2007
Messages
926
Reaction score
167
One other note, I couldn't scp the file because of security issues between the two hosts. I used WinSCP to copy the file to the 16-15.1 system.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
15,083
Reaction score
2,601
Looking at the script, I don't see where you actually import the cdr1313.txt and cel1313.txt into the database? The files are created and at the end erased.
sed -i '/cdr` DISABLE KEYS/r cdr1313.txt' asteriskcdr1615.sql
sed -i '/cel` DISABLE KEYS/r cel1313.txt' asteriskcdr1615.sql
 

Members online

PIAF 5 - Powered by 3CX

Forum statistics

Threads
22,319
Messages
137,021
Members
14,550
Latest member
treimers