Qmail-Scanner-1.24st (st patch)

This is a patched version (ergo unofficial) of qmail-scanner-1.24, that adds some options focused in deal with spam and others features.

Qmail-Scanner (by Jason Haar) is an excellent add-on for Qmail, that enables a Qmail server to scan all gateway-ed email searching for virus and/or Spam. For detailed instructions on how to install and run qmail-scanner visit the home page at http://qmail-scanner.sourceforge.net/, in this page you will only find explanations of the options added by this patch.

I started running qmail-scanner in april 2002, mainly to stop viruses arriving to my users by mail, but since march 2003 the volume of spam mail had increased enormously and my users clamed to block all those messages. So I modified the code of qmail-scanner with the patch from Chris Hine to block (quarantine) spam, based in the score of SpamAssassin, most of my users don't know how to filter messages tagged as spam. And later I added some other little functionalities.

1. Download

It's possible to download the patch (q-s-1.24st-20041101.patch.gz) and apply it yourself, or download a complete distribution (q-s-1.24st-20041101.tgz) already patched.

See the file CHANGELOGpatched to know what is new in this version.

2. To apply the patch

(Skip this step if you have downloaded the distribution already patched)

Untar the file "qmail-scanner-1.24.tgz", cd to the parent directory of the directory "qmail-scanner-1.24" and copy the patch there and gunzip it. (Ok... just do this..)

tar xzf qmail-scanner-1.24.tgz -C /var/tmp/ cp q-s-1.24st-20041101.patch.gz /var/tmp/ cd /var/tmp gunzip q-s-1.24st-20041101.patch.gz

Apply the patch

patch -p0 < q-s-1.24st-20041101.patch

3. Antispam options

You can read in this separate page all the configuration-options of this patched version.
(For detailed instructions on how to install and run qmail-scanner visit the home page at http://qmail-scanner.sourceforge.net/, in this page you will only find information about the specific options of the patch)

./configure ...your options... --sa-quarantine [num] --sa-delete [num] --sa-reject [yes|no]

a) --sa-quarantine [num]    (default 0, no message will be quarantined)

NEW: Now sa-quarantine is a relative value to the SpamAssassin required_hits.

You can set a score in /etc/mail/spamassasin/local.cf (for example 6.5) and SpamAssassin will tag as spam all messages over this score, messages that exceed the "required_hits + sa-quarantine" are quarantined.

Basically what it does is extend the spam checking, so that if a message exceeds a certain configurable spam threshold, the message is quarantined as though it had a virus. Obviously this is only relevant if SpamAssassin is detected.

I check every day the subject of the quarantine messages and I have never seen a false positive over 8 points until 24th december 2003, I got three in one day, two with a score of 8.1 and one with 8.4. And very democratic, one in english, one in italian and one in spanish. Really some people writes down all sort of silly things in Christmas Greetings and SpamAssassin was confused. So, better be ready (how to requeue a quarantined message to the recipient)

The string 'spam' have been added to the "@silent_viruses_array", so no notify will be sent to the sender, as usually is a faked sender. If you don't want this option, edit qmail-scanner-queue.pl and remove 'spam' from the array.

a.1) --sa-forward <user@domain>   (defaults to nothing)

User to redirect spam mails 'being quarantined' for admin purposes...
The mails are redirected almost unmodified to the address set in this option, (an Ip.Guy suggestion) so you can use sa-lern with them.
(i.e. --sa-forward antispam@mydomain.com ).

a.2) --sa-fwd-verbose [yes|no]   (default: no)

Whether to add the X-Spam headers to the forwarded message. Obviously sa-forward must be defined.

a.3) $smaildir

Some people wants to quarantine spam in a different maildir folder than viruses, maybe to run sa-learn.
The default is: my $smaildir="$vmaildir"; You can change it to whatever you want editing qmail-scanner-queue.pl. i.e. my $smaildir="spamdir";

b) --sa-delete [num]    (default 0, no message will be deleted)

NEW: Now sa-delete is a relative value to the SpamAssassin required_hits.

Similar at sa-quarantine but the messages will be deleted. Messages that exceed the "required_hits + sa-delete" will be deleted.

If sa-quarantine is set, sa-delete must be greater.

It is possible to use both, sa-quarantine and sa-delete. For example you can set "required-hits" of spamassassin to 6.5, sa-quarantine to 2.1 and sa-delete to 4.2. Mails with a score over 6.5 will be tagged as spam, over 8.6 will be quarantined and over 10.7 will be deleted (these are my actual settings, but you have to choose your by your experience).

No notify mail will be sent, neither to the admin.

Now with sa-quarantine and sa-delete as relative values you will be able to do a pseudo per user configuration (never tested). The user can set his own required_hits settings, then the admin (you) sets sa-quarantine and sa-delete, so the user could know at what score over his required_hits the mails are quarantined or deleted.
See FAQ n.19 in the official page for details.

c) --sa-reject [yes|no]    (default no)

If you enable sa-reject and sa-delete is properly set, messages with a score higher than sa-delete will be rejected before the smtp session is closed. Otherwise they are just dropped silently. Messages from the LOCALHOST are never rejected.

Be aware that there is no bandwidth saving, but at least the remote smtp server will have to deal with the rejected messages instead of your server.

The remote smtp server will receive a "554 mail server permanently rejected message (#5.3.0)" code. If you want to customize the messages to the remote server (and the remote user, if there is one) you can edit the source of qmail-1.03 and modify the file qmail.c, it is a short file. Just search for the line

   case 31: return "Dmail server permanently rejected message (#5.3.0)";

and change it to what ever you want, for example (Don't remove the first D):

   case 31: return "DWe have reasons to believe this mail is SPAM (#5.7.1)";

and then recompile qmail (make clean ; make setup check ; strip /var/qmail/bin/*).

qmail-smtpd receives an exitcode 31 from qmail-scanner, but you can use one of the exitcodes that you see in the file qmail.c. Be cautious...

You can see some examples of the logs in the mail server and the message that is sent to the remote user (if he is real...) when a mail is rejected.

Stefano Pasquini has pointed me to a little odd situation, he is using this feature and his server is rejecting several mails from his secondary server, which is running by another ISP, this is really no good. To avoid this embarrassing situation you can add a rule in the tcp.smtp file with the enviroment variable SA_ONLYDELETE_HOST, if this variable is defined, spam mails coming through your secondary server will be deleted instead of rejected.

   your.secondary.server.ip:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl",SA_ONLYDELETE_HOST="on"

Don't forget to rebuild the tcp.smtp.cdb database.

127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl",QS_SPAMASSASSIN="on"

d) --sa-delta [num]    (default 0, no tag will be added to spamc_subject)

If you enabled this feature (only works in FAST_SPAMASSASSIN mode) and set $spamc_subject to some text, your users will recieve an indication (HIGH, MEDIUM, LOW) about the score spamassassin gives to the message, in the subject.

If the message has reach a score minor than required_hits (sa_max) plus sa_delta, the messages will be tagged as LOW, in other words the subject will be somethig like this "SPAM *** LOW", assuming that $spamc_subject="SPAM *** ".

A score betwin sa_max+sa_delta and sa_max+2*sa_delta will be tagged as MEDIUM, and if the score is higher than sa_max+2*sa_delta as HIGH.

Be aware that sa_max+2*sa_delta must be lower than sa-quarantine, otherwise it won't never catch any message.

You can edit qmail-scanner-queue.pl and change this in the sub spamassassin to whatever you want.

e) --sa-subject <"some text">    (default to nothing)

This is an alternative way to set $spamc_subject to some text, for example "SPAM *** ". Be sure that is better to tag the subject, of spam messages (only works in FAST_SPAMASSASSIN mode), through qmail-scanner than with the rewrite_subject of SpamAssassin.The input must be quoted.

f) --sa-alt [yes|no]    (default no)

This is an alternative subroutine to call SpamAssassin. It ALWAYS works in FAST_SPAMASSASSIN mode, and it would be a little bit faster because it doesn't create a tmp_file and neither pass the '-u' option to spamc (but you will find the code commented inside qmail-scanner-queue.pl in the routine sub spamassassin_alt, uncomment it if you need spamassassin sql per user settings).

When I said above ALWAYS I mean ALWAYS, sa-alt sets the spamc_options by itself so if you want to run in VERBOSE_SPAMASSASSIN mode or you want to use the 'sql per user preferences' for spamassassin, you have to disable this option and run the 'standard spamassassin' routine.

This option should be used with the following two options.

f.1) --sa-debug [yes|no]    (default no)

If sa-alt is enabled and you enable this option, you will have a log of the tests and scores from SpamAssassin in qmail-queue.log. And these score and tests will be also added to the notifies sent to the admin.
I was looking for a way to control how SpamAssassin was working, and this is the reason for that I wrote the alternative subroutine to connect to SpamAssassin.
If you enable add-dscr-hdrs you will get the process number and then you can do a grep by the process number in qmail-queue.log and debug what happened with a message.
Don't worry, you don't need to reconfigure qmail-scanner to switch from one subroutine to the other, just edit qmail-scanner-queue.pl and disable/enable sa-alt (sa_alt).
Want to see the configuration of SpamAssassin and a sample of the logs?

f.2) --sa-report [yes|no]    (default no)

If sa-alt and sa-debug are enabled the X-Spam-Report header will be added to the messages enabling this option.
Notice that you are still running in FAST_SPAMASSASSIN mode...

g) SA_SKIP_MD   ( SA skip MAILER-DAEMON )

This is not a configuration option (yet another option for Stefano Pasquini), this is a switch inside the code that you can enable or disable when you need it. Set to something different from zero to enable it.

Supposing that a spammer drops in the net several tens of thousands mails with a random from address like abxtyicj@yourdomain.com, and then in a few minutes your mail server will receive something like 3.800 messages from MAILER-DAEMON because some user from some server is unknown.. Well, you have to deal with all those messages quickly but SpamAssassin spends some seconds for each message, so your server will be on his knees. In this case you can edit qmail-scanner-queue.pl and set SA_SKIP_MD to '1', qmail-scanner will skip SpamAssassin for messages 'From: MAILER-DAEMON' and empty Return-Path, but the antivirus scanners will always check the messages.

I don't think that is a good idea to have it always enabled.

h) --sa-socket <path to spamd socket>   (defaults to nothing)

Actually the configure script can automatically discover if spamd is running in unix-socket mode, but, if for some reasson the socket couldn't be found properly you can set the path with this option. (i.e. --sa-socket /var/run/spamd).

From my test over ten thousand mails, spamd is 7,8% faster running with unix-socket.

4. Miscellaneous options

a) --qs-group <usergroup>    (default: same as qs-user)

Group of the user that Qmail-Scanner runs as. This option allows you to install qmail-scanner-1.24 over an old installation (1.1x) where the user was "qmailq" and the group "qmail". This will decrease the security level, but qmail itself is already heavily compartmented. (This option is only used during the install process).

b) --minidebug [yes|no]    (default: yes)

This option only logs important information to qmail-queue.log, that give me a sense that how it is going up.

If you enable debug, minidebug is automatically disabled.

c) BMC_WHITELIST="on"

Set this enviroment variable in tcp.smtp to disable BAD_MIME_CHECKS for some servers. It's a little bit hard to mantain...

d) --virus-to-delete [yes|no]   (defaults to "no")

Enable this option if you want to delete some viruses (i.e. mydoom) without notifying anyone. If you don't enable it now, you can later edit qmail-scanner-queue.pl and add the virus you want to the list virus_to_delete.

e) --scanners-per-domain [yes|no]   (defaults to "no")

Enable or disable the domain-wise mode, each user/domain will have a customized @scanner_array. If the user/domain haven't a custom @scanner_array, qmail-scanner will fall to the @scanners_default array.

You have to edit the file 'scanners_per_domain.txt' and configure there your domains, you will find some instructions inside the file. More info...

5. Cosmetic options

a) --admin-fromname <"From Name">    (default: "System Anti-Virus Administrator")

As its name says, this option sets the from name for the emails reports sent by qmail-scanner, I was annoyed of edit and change it on every new installation. The input must be quoted.

b) --dscr-hdrs-text <"Descrip-Headers-Text">   (defaults: "X-Qmail-Scanner")

Well, the same as previous...

c) log-report.sh

This script is installed in the qmailscan directory and does a quick statistic from the qmail-queue.log files, you can send a mail after rotating the logs....

  ./log-report.sh qmail-queue.log.1.gz

   2841 W32/Netsky-P
    430 W32/Mabutu-A
    218 W32/MyDoom-O
    142 W32/Netsky-Q
     70 W32/MyDoom-A
     57 W32/MyDoom-H
     36 W32/Netsky-N
     12 W32/MyDoom-N
     11 W32/Mydoom-T
     11 W32/MyDoom-S
      4 W32/MyDoom-AG
      3 W32/Zafi-B
      2 W32/NetskyP-Dam
      2 W32/Mydoom-F
      2 W32/Lovgate-V
      2 W32/Flcss
      1 W32/Torvil-A
      1 W32/Parite-B
      1 W32/Netsky-C
      1 W32/Bagle-AU


  11173 rejecting
    596 tagging
  

6. Examples

This will be an example of installing over a previous 1.1x installation, obviously the mailbox "antivirus@mydomain.com" should exist... The required_hits in the file /etc/mail/spamassassin/local.cf is '6.5'. If you're upgrading from 1.1x, don't try the manual installation, lets the configure script do its job.

./configure --qs-user qmailq \ --qs-group qmail \ --domain mydomain.com \ --admin antivirus \ --admin-fromname "Antivirus MYDOMAIN" \ --add-dscr-hdrs yes \ --dscr-hdrs-text "X-Antivirus-MYDOMAIN" \ --ignore-eol-check yes \ --redundant yes \ --max-zip-size 50000000 \ --virus-to-delete yes \ --block-password-protected yes \ --sa-quarantine 2.1 \ --sa-delete 4.2 \ --sa-reject yes \ --sa-subject "SPAM *** " \ --sa-delta 0.5 \ --sa-alt yes \ --sa-debug yes \ --sa-report yes \ --sa-socket /var/run/spamd [ --install ]

For an standard installation (new or upgrade) with the user qscand, (first create the user) and then the options below would be enough:

groupadd qscand useradd -c "Qmail-Scanner Account" -g qscand -d /var/spool/qmailscan -s /bin/false qscand

./configure --domain mydomain.com \ --admin antivirus \ --admin-fromname "Antivirus MYDOMAIN" \ --add-dscr-hdrs yes \ --dscr-hdrs-text "X-Antivirus-MYDOMAIN" \ --ignore-eol-check yes \ --redundant yes \ --max-zip-size 50000000 \ --virus-to-delete yes \ --block-password-protected yes \ --sa-quarantine 2.1 \ --sa-delete 4.2 \ --sa-reject yes \ --sa-subject "SPAM *** " \ --sa-delta 0.5 \ --sa-alt yes \ --sa-debug yes \ --sa-report yes \ --sa-socket /var/run/spamd [ --install ]

I hope these options will be useful for you as they are for me. There isn't a specific mailing-list for this version, you can reach the official qmail-scanner-general mailing-list, you will find a lot of good stuff there.

Thanks to Jason for this very very good tool.

Thanks to Chris for the spamassasin quarantine patch, all my users are very happy since the patch was installed blocking tons of spam.

Salvatore Toribio

20041101

No warranty, expressed or implied, etc, etc, etc...