Howto delete multiple files in Linux and FreeBSD / How to deal with “Argument list too long” error while deleting many files in directory

Wednesday, 7th April 2010

Linux has some Limitations on the number of files you can delete within a directory, therefore if you try to delete let’s say 100000 files with a quarantine mails from spamassassin.
In that case you are about to face an error Argument list too long . The amount of files you can delete in Linux is tied with something specified by a file:
/usr/include/linux/limits.h
This limitation is a limitation caused by kernel_limits. In order to check the limitation on your Linux distribution, you have to execute the command:

egrep ARG_MAX /usr/include/linux/limits.h

You should receive a result on most Linux distrubutions similar to:
#define ARG_MAX 131072 /* # bytes of args + environ for exec() */

The 131072 is actually a default limitation on Debian GNU/Linux as well.The reason for the error is that the the maximum number (in bytes) of the arguments to a command could be equal max to the ARG_MAX defined in the limits.h.
For instance rm -f * in a directory with 40000 fileswould be evaluted as rm -f file1 file2 file3 … file40000. Therefore at a certain point the maximum limitation of 131072 bytes long for arguments or 128KB is about to be reached and then the command let’s say ls * would refuse to list the files in the directory showing up the annoying Argument list too long error.
There are a couple of ways to deal with that unpleasant situation.

1. You can use the linux find command to delete the files, you have to execute after changing dir (cd) to the directory where the multiple files are located:
find . -exec rm -fr {} ; 2. Second approach to the problem is passing the xargs command to find .
For instance execute the command:

find . -name "*" -print | xargs rm

3. In FreeBSD to get around the “Argument list too long” problem”, in bash shell you have to execute:

for files in *.*; do rm -f $files; done

4. Another possible way is to increase the ARG_MAX value in limits.h though this approach in my personal belief could have a negative impact on some productive servers, therefore it’s not a recommended.
Yet if you desire to do so simply edit /usr/include/linux/limits.h and change the ARG_MAX to your value of choice.

Share this on:

Download PDFDownload PDF

Tags: , , , , ,

3 Responses to “Howto delete multiple files in Linux and FreeBSD / How to deal with “Argument list too long” error while deleting many files in directory”

  1. Lich King says:
    Firefox 4.0.1 Firefox 4.0.1 Windows Vista Windows Vista
    Mozilla/5.0 (Windows NT 6.0; rv:2.0.1) Gecko/20100101 Firefox/4.0.1

    i had a cron misconfig and it was saving a fuck load of useless mail in Maildir/new

    that freebsd command helped me getting rid of that!

    View CommentView Comment
  2. Arnold Galetti says:
    Firefox 3.5.3 Firefox 3.5.3 Windows XP Windows XP
    Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3

    Hello, I Jacob from UCT South Africa, i am doing a research on this field for my Master’s project. I would be glad if you would permit me to cite this article in my dissertation.

    View CommentView Comment

Leave a Reply

CommentLuv badge