FOOD FOR THOUGHT Asterisk bombs, ipbx 13-12, ubuntu 14

[chris_c_]

Honestly, I haven't seen any problems with apt-get and the current status setup has been in place for a couple of years. We add stuff with apt-get all the time. What am I missing?

Sometimes apt-get needs to shutdown or start services, for example, the script that upgrades the system package udev. To do this, it issues the command to stop, and waits for udev to actually stop by using the status binary. This will fail and hang if the status command has been replaced by ipbx! My system got stuck on upgrading udev and would've stayed waiting forever, until I figured out what was actually happening, ipbx replaced "status" and broke that operating system feature!

I spent a lot of time wrestling with this when we first set up the Ubuntu build. Here's the problem, and maybe the two of you have an answer. If you do a "normal" /sbin/halt or shutdown -h now, the important stuff doesn't get shutdown properly, i.e. Asterisk, MySQL, and Apache. This led to all sorts of corruption issues if something happened to be going on when the halt command was executed. That's why we initially went to the reboot and halt scripts in /usr/local/sbin to make sure things got closed down gracefully. It's easy enough to change status to statusmenu or some other name in all the builds, but I don't want to introduce corruption problems in the process.

If we can iron out this kink, I'm more than happy to revisit the status naming issue so long as we implement it across all the platforms.

In the meantime, here's the fix but I strongly recommend using the halt and reboot scripts to stop and restart your server!

mv /sbin/statusU /sbin/status
mv /usr/local/sbin/status /usr/local/sbin/statusmenu
sed -i 's|status |statusmenu |' /root/.profile

The reason the system gets hung up during a "standard" call to "shutdown" is because the operating system calls "status" during the shutdown, to check and see if various services are going down as expected, it needs to wait on that before it does a halt! And instead of receiving expected responses from status, the ipbx status runs a "dialog" binary to display the color text screen, which you never see because the operating system redirects the output to itself, and that dialog binary waits forever for an "enter" which the operating system never hits!

 

[wardmundy]

Sorry, I'm missing something. We run a dialog binary all the time. That's what status does. Saying shutdown hangs forever because users can't see the dialog when Ubuntu redirects the output to itself REALLY doesn't solve the shutdown problem. We need a stable procedure for folks who depend upon their phone systems in the real world.

 

[chris_c_]

Sorry, I'm missing something. We run a dialog binary all the time. That's what status does. Saying shutdown hangs forever because users can't see the dialog when Ubuntu redirects the output to itself REALLY doesn't solve the shutdown problem. We need a stable procedure for folks who depend upon their phone systems in the real world.

Right. Rephrasing it.
The ipbx version of status calls dialog. That works, dialog shows the color screen.
The operating system binary apt-get, as well as the OS shutdown binary, both call status, and wrongfully gets the ipbx version of status, which calls dialog, which fails to return the expected text to the original caller, and the original caller hangs forever waiting for the expected text saying a particular service has safely stopped.
Solution: ipbx version of status needs to be renamed something else. I suggested pbx_status , you statusmenu , either one. restore the OS status binary, and suddenly both "shutdown", and apt-get for upgrading OS services, work again.

 

[wardmundy]

I used the commands shown above to restore Ubuntu's status binary but it still fails.

 

[chris_c_]

I used the commands shown above to restore Ubuntu's status binary but it still fails.

Do you mean shutdown still fails ?
You probably still have a dialog binary still running in the background waiting on input and blocking things from shutting down.

ps aux | grep dialog
sudo kill (pid number of dialog)
sudo shutdown -Ph now

 

[wardmundy]

Dialog wasn't running. shutdown -Ph now works, but it's almost instantaneous. There is no way it is gracefully shutting down all running services. Might as well pull the plug out of the wall. Sorry, but that won't work for an Asterisk server that's dependent upon FreePBX and MySQL.

 

[wardmundy]

chris_c_ said:

"My system got stuck on upgrading udev and would've stayed waiting forever, until I figured out what was actually happening, ipbx replaced "status" and broke that operating system feature!"

Status didn't get replaced. It was renamed. But that's why I inquired about your server. You didn't answer my questions. If you had followed the tutorial to install Incredible PBX, there is no way you would have needed a udev upgrade. That's the first step in the install process and occurs long before anything associated with Incredible PBX is ever loaded. Our builds are all designed for installation on a fresh (upgraded) OS. If you're doing something different, that probably explains your curl problem as well.

 

[chris_c_]

Dialog wasn't running. shutdown -Ph now works, but it's almost instantaneous. There is no way it is gracefully shutting down all running services. Might as well pull the plug out of the wall. Sorry, but that won't work for an Asterisk server that's dependent upon FreePBX and MySQL.

What shutdown does is stop all the services through their scripts in standard location /etc/init.d

service asterisk stop
service apache2 stop
service mysql stop

If that's not good enough then probably the amportal script should be called from or moved to /etc/init.d so as to more gracefully shut those services down.
Bottom line is, a linux system needs to be able to be shutdown via the shutdown command - this is industry standard and it's basic requirement - all cloud control panels call "shutdown -Ph now" as they should. So the scripts need to hook into that and do their stop through /etc/init.d

 

[chris_c_]

chris_c_ said:


Status didn't get replaced. It was renamed. But that's why I inquired about your server. You didn't answer my questions. If you had followed the tutorial to install Incredible PBX, there is no way you would have needed a udev upgrade. That's the first step in the install process and occurs long before anything associated with Incredible PBX is ever loaded. Our builds are all designed for installation on a fresh (upgraded) OS. If you're doing something different, that probably explains your curl problem as well.

Right, splitting hairs with semantics here! status was effectively replaced with the ipbx status binary, while the original ubuntu binary was backed up to statusU.
I followed the tutorial on the blog post to install ipbx. The point is, updates to udev (and other system packages which require a functioning ubuntu status binary) can come along at any time, and should be installed. This requires a functioning ubuntu status binary, named status. ipbx really should not grab that name status, thereby breaking standard, native OS security/bugfix update and shutdown features.

 

[wardmundy]

But that's not what you said you did.

 

[chris_c_]

But that's not what you said you did.

What, sorry? I ran the installer script IncrediblePBX13-12U.sh, which replaced status by putting its own status in the path and renamed the OS status so it wouldn't be invoked.

(317.) mv /sbin/status /sbin/statusU

 

[wardmundy]

What I understood you to say was that you were updating udev and it hung your system. That wouldn't have happened running the installer. Doesn't really matter. Was just trying to track down why you had a problem and THAT would not have been caused by running the Incredible PBX installer with its upgrade and update routines which occur before any other changes are made to Ubuntu. I've documented how you can fix this which I assume you already knew. Frankly, yours is the first complaint in more than two years and I'm not yet persuaded that the cure isn't worse than the problem. But we'll do some more testing. Thanks.

 

[chris_c_]

We should integrate the amportal script as a proper service in ubuntu/debian, instead of just a script that sits in /usr/local/sbin and gets called when the user manually invokes another script.
This way as a proper standard service, amportal will be subject to the settings of runlevel, and rules of startup order, and shutdown order.
Here's some code to add to IncrediblePBX13-12U.sh and try out:

#Get amportal (freepbx) to start automatically on boot and stop automatically on shutdown.
 
    echo '#!/bin/bash' > /etc/init.d/amportal-start
    echo '/usr/local/sbin/amportal start' >> /etc/init.d/amportal-start
    chmod +x /etc/init.d/amportal-start
    update-rc.d amportal-start start 99 2 3 4 5 .
 
    echo '#!/bin/bash' > /etc/init.d/amportal-stop
    echo '/usr/local/sbin/amportal stop' >> /etc/init.d/amportal-stop
    chmod +x /etc/init.d/amportal-stop
    update-rc.d amportal-stop stop 10 0 1 6 .
 
    /etc/init.d/asterisk stop
    update-rc.d -f asterisk remove
 
    /etc/init.d/apache2 restart
    amportal kill
    dahdi_genconf -F
    /etc/init.d/dahdi restart
    amportal start

"sudo amportal stop" takes 3.5 seconds on a asterisk that has no currently active calls.
"sudo amportal start" takes 7 seconds.
How many seconds does it take your system to complete this with no calls active?? Should be around 4-5 seconds

sudo shutdown -Ph now

 

[Joe Lones]

@chris_c_ I seem to have the same errors in my log as you as per your OP, any resolution to it?

This is my issue http://pbxinaflash.com/community/threads/clean-install-asterisk-crashes.18858/#post-118869