TIPS Trunks wont reconnect till reboot

[just_some_random_dud]

My first time posting here, I just took over for someone else and we have PIAF boxes at several sites on lots of different hardware with almost no internal documentation. It seems like a great system but I don't even know where to start with this problem so if this is not the right place I apologize.

The issue we are having is that occasionally our sip trunks will lose connection to one pbx or another and stay disconnected until the system is rebooted. I believe we are using the green flavor. As a complete and total beginner I have no idea where to start. Is there any way to determine what is happening? Or anyway to force the system to try to reconnect the trunk automatically if it disconnects? Assume that I don't know what I am doing and that I am doing everything wrong.

 

[just_some_random_dud]

lqqqqqqqqqqqqqqqqqqqSYSTEM INFORMATION *VERIFIED*qqqqqqqqqqqqqqqqqqqqqk
x Asterisk = ONLINE | Dahdi = ONLINE | MySQL = ONLINE x
x SSH = ONLINE | Apache = ONLINE | Iptables = ONLINE x
x Fail2ban = ONLINE | Internet = ONLINE | Ip6Tables = ONLINE x
x Disk Free = ADEQUATE| Mem Free = ADEQUATE| NTPD = ONLINE x
x SendMail = ONLINE | Samba = OFFLINE | Webmin = ONLINE x
x Ethernet0 = ONLINE | Ethernet1 = N/A | Wlan0 = N/A x
x x
x PIAF Installed Version = 2.0.6.4 under *HARDWARE* x
x FreePBX Version = 2.11.0.25 x
x 2.11 x
x Running Asterisk Version = 11.7.0 x
x Asterisk Source Version = 11.7.0 x
x Dahdi Source Version = 2.8.0.1 x
x Libpri Source Version = 1.4.14 x
x IP Address = 192.168.1.243 on eth0 x
x Operating System = CentOS release 6.4 (Final) <> x
x Kernel Version = 2.6.32-358.2.1.el6.i686 - 32 Bit x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

 

[just_some_random_dud]

The trunks seem to get disconnected almost at random, with some sites going months without issue and other sites dropping once a week. We have this problem on at least 4 or 5 different hardware builds (all intel atom or intel corei3's with ssd's in *most*) We also have issues regardless of ISP.

I am not looking for a miracle answer, just guidance on where to start.

 

[rjaiswal]

Hi!

And welcome to the forums!

I have a couple of questions.

What kind of trunks are dropping? SIP?

Are they with the same carrier?

What kind of firewall is between the pbx and the Internet?

 

[just_some_random_dud]

Sip trunks, the carrier is Voxox (formerly Telecentris) for all trunks. The firewalls are all pfSense.

 

[rjaiswal]

Ok.

One more question. How are you authenticating to Voxox? Registration, or IP authentication?

 

[just_some_random_dud]

Right now I am waking up in the morning and calling every site at 6am to make sure the trunk didn't disconnect during the night. Again this seems to be happening with *most* of our pbx's and I usually have to reset one of them once a week. I have currently been unable to determine any way to predict when or where this will occur or any common denominator. We have some sites go months without issue and others that have more frequent outages. It wouldn't be a big deal except that we don't really have a system in place to monitor these connections so basically we don't know that there is a problem until it is reported by an end user and frequently that might mean they have missed hours (or days if it is the weekend) of incoming calls. And then we have to remote in and reboot the PBX. So I am looking for either a way to monitor and automate this, or a better way to make the trunks recover gracefully or even better a way to fix the issue. but once again I am brand new to Piaf and asterisk in general and have come from the world of windows system admin........So any guidance would be great. I don't mind getting my hands dirty.

 

[just_some_random_dud]

Ok.

One more question. How are you authenticating to Voxox? Registration, or IP authentication?

Registration.

username=xxxxxxxxxx
type=friend
secret=xxxxxxxxxxx
nat=no
insecure=port,invite
host=69.xxx.xxx.xxx
dtmfmode=rfc2833
disallow=all
context=from-trunk
canreinvite=nonat
allow=ulaw

 

[rjaiswal]

Ok.

I would first call the carrier and see how you are authenticating with them. If it's registration, I would try switching it to IP authentication.

Now this would only work, if your sites have a static IP address.

Also, when using IP authentication, you need to forward sip and RTP ports to your pbx.


I would try this on one of the more problematic sites first.

Voxox should have instructions on how to set this up.

Now if your sites have dynamic addresses, I would setup the port forward on pfsense for sip and RTP, just to rule out any uPnP issues.


Also, take a look at this thread for setting up email monitoring of trunks.

http://pbxinaflash.com/community/index.php?threads/email-alert-on-trunk-failure.1968/

 

[Chris Sweeney]

While everyone comes up with an idea to help you, one thing I can share is I doubt you need to actually reboot the machine. Have you tried just reloading asterisk? Usually that will cause the trunks to re register and save you having to do a complete reboot. I'm not familiar with your carrier so I'm not going to be any help there most likely but I wonder if having the trunks "ping" would help keep them up. I might start by adding
qualify=yes
to your trunk settings and see if that helps keep the trunks up.

 

[just_some_random_dud]

to be clear also, the piaf system is always up and the phones are online and connected and I am able to log directly into it and look around just the status page shows no trunks connected until I reboot the box.

 

[just_some_random_dud]

thank you for the tremendously helpful responses, I will try all of these things and let you know.

 

[wardmundy]

For the short term until we can solve the problem, you might want to add a cron job at each problem site to restart Asterisk at 5 a.m. so you can sleep later.

1. Just build a script called /root/restart.sh that looks like this:

#!/bin/bash
amportal restart

2. Make the script executable: chmod +x /root/restart.sh

3. Then add a line to /etc/crontab with a command like this:

echo "0 5 * * * root /root/restart.sh >/dev/null 2>&1" >> /etc/crontab

 

[mbellot]

Ward - Thanks for the script, it seems rather timely since my PIAF box seems to have developed the same issue. I'm seeing a random trunk drop registration and not re-connect unless I do a reload (not reboot).

I'll be adding your script to my system to see if it fixes the problem, but I'm wondering if anyone has any idea why this started happening and can we figure out a real fix instead of the script band-aid.

I have two Voip.ms trunks and a third with someone else, but all three seem affected.

Just for the record, here is my system info screen.

PBX in a Flash GREEN Status Program
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
  lqqqqqqqqqqqqqqqqqqqSYSTEM INFORMATION *VERIFIED*qqqqqqqqqqqqqqqqqqqqqk
  x  Asterisk  = ONLINE  | Dahdi    = ONLINE  | MySQL    = ONLINE    x
  x  SSH        = ONLINE  | Apache    = ONLINE  | Iptables  = ONLINE    x
  x  Fail2ban  = ONLINE  | Internet  = ONLINE  | Ip6Tables = ONLINE    x
  x  Disk Free  = ADEQUATE| Mem Free  = ADEQUATE| NTPD      = ONLINE    x
  x  SendMail  = ONLINE  | Samba    = OFFLINE | Webmin    = ONLINE    x
  x  Ethernet0  = ONLINE  | Ethernet1 = N/A    | Wlan0    = N/A      x
  x                                                                    x
  x  PIAF Installed Version  = 3.6.5 under *HARDWARE*                  x
  x  FreePBX Version          = 2.11.0.31                              x
  x 2.11                                                                x
  x  Running Asterisk Version = 11.5.1                                  x
  x  Asterisk Source Version  = 11.5.1                                  x
  x  Dahdi Source Version    = 2.7.0.1                                x
  x  Libpri Source Version    = 1.4.14                                  x
  x  IP Address              = 192.168.0.10 on eth0                    x
  x  Operating System        = PBX in a Flash release 3.6.5 (PIAF3)    x
  x  Kernel Version          = 2.6.32-431.5.1.el6.i686 - 32 Bit        x
  mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

 

[just_some_random_dud]

unfortunately this still seems to be occurring for us, we have had it happen several times in the middle of the day, I stopped and started the asterisk service but It did not come up until I rebooted the system. We are in the process of adding secondary trunks from vitelity so see if this issue has anything to do with our provider and I will report back.

 

[just_some_random_dud]

Can confirm now that when this issue happens it affects both our voxox AND vitelity trunk, they both go offline and wont reregister until a reboot. there is a good chance I am not reloading asterisx correctly, but stopping the service and starting it again seems to have no affect.

 

[MGD4me]

Does your ISP provide a fixed IP address to your systems, or does it assign an address via DHCP? Is it possible that when the DHCP lease times out, a totally different IP address is issued, dropping your trunks in the process? Check to see if your WAN address is changing. Just a random shot in the dark...

 

[RPi-Fan]

It's not clear if this is the source of your problem, but it may be worth a try...

By default, if Asterisk receives a 403 response to a registration, it does not continue retrying until Asterisk is restarted. This behavior can be changed at:

Settings -> Asterisk SIP Settings -> Other SIP Settings

by adding:

register_retry_403 = yes

This will cause Asterisk to treat 403 responses to registrations as if they were 401 responses and continue retrying according to normal retry rules.

 

[Chris Sweeney]

Can confirm now that when this issue happens it affects both our voxox AND vitelity trunk, they both go offline and wont reregister until a reboot. there is a good chance I am not reloading asterisx correctly, but stopping the service and starting it again seems to have no affect.

What type of firewall are you running behind? Could this be a NAT translation timeout issue?

 

[just_some_random_dud]

We are running a pfsense firewall.