Advertenties

No space left on device – running out of Inodes

One of our development servers went down this weekend. Problems started with a small PHP script that claimed that claimed “No space left on device”. Nagios however reported 32% usage of space, so that couldn’t be it?

If you ever run into such trouble – most likely you have too many small or 0-sized files on your disk exhausting all available Inodes.

Try this;

[code]$ df

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda 33030016 10407780 22622236 32% /
tmpfs 368748 0 368748 0% /lib/init/rw
varrun 368748 56 368692 1% /var/run
varlock 368748 0 368748 0% /var/lock
udev 368748 108 368640 1% /dev
tmpfs 368748 0 368748 0% /dev/shm[/code]

Now check for the inodes
[code]$ df -i

Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/mail1-root
610800 597995 12805 98% /
tmpfs 256574 4 256570 1% /lib/init/rw
udev 252892 463 252429 1% /dev
tmpfs 256574 1 256573 1% /dev/shm
/dev/xvda1 124496 222 124274 1% /boot
[/code]

If you have IUse% at 100 or near, then huge number of small files is the reason for “No space left on device” errors.

Now run a search on the filesystem to find them
[code]$ for i in /*; do echo $i; find $i |wc -l; done[/code]

And now you can delete this directory!

Advertenties

Geef een reactie

%d bloggers liken dit: