Debugging Memory Problems

I am running Purple release on the Acer Revo Box recommended in Nerd Vittles. I contnue to see creep in the used App Memory. We have a total of three users on the system and peering with threee service providers. The only thing that seems to help is rebooting the host system. Is anyone else seeing this? Anyone have a fix?

the easiest way to tell if you have (or rather, don't have) a problem: get a shell and do 'free -m'. on mine it looks like:

root@pbx:~ $ free -m
total used free shared buffers cached
Mem: 2026 1383 643 0 808 299
-/+ buffers/cache: 275 1750
Swap: 760 0 760

If the increasing usage is in the 'cache' section, it is not an issue.

Rebooted - here's data

total used free shared buffers cached

Mem: 883 468 414 0 25 194
-/+ buffers/cache: 249 633
Swap: 760 0 760

PIAF shows app mem at 29%

Will check later today for values.

PS Ward, I know Linux managed the memory, but this was NOT experienced with last version running under Centos 5.5 32 bit. Only happened since purple release was loaded

Since App memory is now reaching 91% we also see Swap memory climbing and it hits 89%. Having to reboot is annoying.

Not a fix but a potential workaround for you...

To save power on my pbx I run a cron job each night that shuts down the computer.

Code:

shutdown -h now

The computer powers up back automatically each morning using the Bios RTC setting to power on each day at the same time. If you have a service that calls go to voice mail or forward to another number, the alternate handling kicks in... In my case you get busy... You can either wait until morning or if its an emergency you probably have my cell number.

In your case you could shutdown for say 5 minutes and then power back on. Run the cron job at say 11: 59 pm, set the bios to power on at 12:04 am.

Not necessary for you but I also use Smart Strip LCG3 Energy Saving Surge Protector to power off all the ancillary devices. Note that this may not work with a real low power computer, it requires a minimum difference between power drawn when on versus when off.

Tom,
The box hit 89% app mem and 81% SWAP (from FreePBX stsus screen) last night and call processing was not functional. All calls (internal and external) went to VM defined as last termination location in dialplan.
I rebooted 11 hours ago (unfortunately forgetting to run statustofile first). Call processing was restored.
11 hours later with no calls processed/attempted APP memory is at 60% and SWAP 0%. Current statustofile.txt is attached.


I will run it when higher levels are reached

Thanks, that's food for thought

Have you tried "top" and checked to see what is hogging your memory? Add swap size with the commands f then p.

Eugene, yes I have and it is the process associated with command "asterisk"

Hi

You may get more idea of what is happening by turning up the verbosity, and watching the full log.

Code:

rasterisk -vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
tail -f /var/log/asterisk/full

This may show something not quite right.

Joe

Joe,
Thank for your suggestion
The following series of messages repeat constantly

[2010-12-03 15:14:37] ERROR[3111] res_jabber.c: JABBER: encryption failure. possible bad password.
[2010-12-03 15:14:37] ERROR[3111] res_jabber.c: aji_act_hook was called with out a packet
[2010-12-03 15:14:37] WARNING[3111] res_jabber.c: Parsing failure: Hook returned an error.
[2010-12-03 15:14:37] WARNING[3111] res_jabber.c: JABBER: Got hook event.
[2010-12-03 15:14:37] WARNING[3111] res_jabber.c: JABBER: socket read error

I didn't know we used JABBER in asterisk or Free PBX or PIAF.

When I go to WEBMIN and look at active processes i get the data shown in attached file.

I have not configured these trunks, but they are disabled in Free PBX. I assume from your note that disabling them there is not sufficient and that I must modify the configuration files as you suggest. May I inquire where this is documented - I must have missed it.

Is it possible that something else is listening on the jabber port?
The socket read error made me think of this.

FWIW, My purple build has been running fine since I built it a couple of weeks ago.

Obviously your asterisk process is using significant RAM.

If you're not using GV.. (outbound is broken at this writing) you could unselect with make menuselect jabber, chan_gtalk and and re make.
Not sure thats an option.

did you do as darmock suggested and choose not to load jabber and chan_gtalk? If that works its obviously easier.

This may be WAY off base but my gigabit ethernet eth0 was a bit flaky (skge) so I've been running on my eth1 which is an nforce 100mbit. Likely an irrelevant detail.

I'm hoping you get it resolved soon.

A top from my system sorted by resident memory consumption.

Tom, I wasn't complaining, I just wanted to learn from my mistakes. I am visually impaired so sometimes I miss things in documentation. I have made the change and the messages have gone away. I will monitor the APP MEM and SWAP percentages.