TUTORIAL HOW-TO Find Big Files

[LesD]

Running in a VM. I see that the disk is 70% full (14GB out of 20GB). I would not expect the system to be larger than 4 GB or so.

Which folders should I check looking for large deletable files?

I have check for automated backup files but there seem to be none.

Thanks

 

[randy7376]

LesD

You can use du -sh <path> to show space in-use. A good place to start: du -sh /var/log

This will show you how much space is used in /var/log. You can then go into /var/log and do du -sh * | sort -n and it will sort the output putting your largest file sizes at the bottom.
For command usage, see man du

I'd dheck /var/lib/asterisk also.

As always, someone may have a better way of accomplishing the same thing.

 

[bobkoure]

I recently had xmpp (google voice) write debug messages like crazy to /tmp/refs.
I'd changed something in my gapps which invalidated my app login and that file got big.
Seems to me I used something like du -sh / | sort -n to find it. Don't trust my linux-fu, just mentioning this as you might have something in /tmp as well.
I fixed my gvoice app password, had a connected trunk, but debug file continued to grow quickly. Deleting the trunk stopped the file. All this on a RasPi. I plan to move to faster hardware soon so I didn't dig any deeper.

 

[wardmundy]

I use this to find large files over 10MB on an entire drive. Change / to any subdirectory and 10000 to any size desired:

find / -type f -size +10000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

 

[LesD]

Thank you - found the culprit yesterday with du (did it before I got the post from Ward)

File called messages in var/log was over 10 GB!

It was not an accumulation of old messages as there were proper rotated copies from earlier days.

So it all must have been from Monday. Strange.

But there was an other strange thing Monday - maybe connected.

I noticed that the cpu bar was red at 100% (though the average was just showing over 1%)

I restarted the VM but after about 30 seconds the 100% came back though again the average showed only about 1.6%.

I closed down the VM and gave it 2 cores instead of 1 (could not think of anything else to try)

Restarted the VM and problem solved! Two hours after restart it all looked OK and the messages file has not yet been recreated since I deleted it.

So I restarted the VM with a single core and it was back at 100%

File messages was 25 MB after a few minutes!

The file is full of the following line (with only time changing):

Jan 26 20:22:12 pbxrand atd[26236]: File a000130169adf2 is in wrong format - aborting

I found the file in <var/spool/at> dated yesterday 10am with a partner of the same name but with an = instead of the a at the front of the name.

Both of length zero. Deleted both, shut down the VM, set it back to 1 core and now it all looks happy. Checked after 11 hours and all is well.

I generally do not look at this machine so it was most fortunate that I spotted to red bar, else I suppose the system would have been dead within an other few hours with a full disk.

Anyone know what that file is about and why the difference in behaviour between 1 and 2 cores?

As to the damaged file there is maybe a clue. The server on which the VM runs was force shut down yesterday morning so that is the most likely cause of the file damage.


I still had the disk showing at 75%. Needed a re-booting to bring it down to 24%.

Running PIAF Green-3.6.5.

 

[Dave Gray]

Files in /var/spool/at are used to run commands at a specified time (read the docs on at by typing man at). The at command is handy when you need to run something at a specified time - as a one off. If you need to run something regularly, then you use cron.

As to the disk space, when you remove (rm) a file, the file isn't deleted until all processes that have that file open, are complete. It's a common trick, when you need an intermediate, temporary file, to create one, open it, then delete it (while it is still open.) You can read and write it as you need, then when your program finishes, it vanishes. You'll see this when you have a log file, for instance, that is huge, so you delete it. But, the log daemon still has it open, so all the space is still used until the daemon restarts. Worse, you won't see any new messages until it restarts, since you cannot see the log file any more.

(This is what 30+ years of Unix experience does to ya...)

 

[wardmundy]

Learn something new every day! Thanks, Dave Gray.