Posts Tagged ‘spamassassin’

How to solve crashing spamassassin spamd service preventing QMAIL mail server to properly deliver mails / Setting spamd work via daemontools

Monday, November 12th, 2012

How to solve crashing spamd, script to restart spamd on failure, set spamd to run via daemontools, configure spamd to be restarted from monit service

On my home router configured qmail install, I have recently noticed I receive e-mails only sent via contact forms on the few websites hosted there. I'm subscribed to Debian newsletter, as well as usually receive about 10 emails and few spam mails every day, so after a few days of reduced emails on my email (receiving only e-mail notification about  blog issued comments), I logically suspected something is not properly working with the qmail installation.
My first logical guesses was the usual Qmail problems I've previously experienced through the years, earlier I blogged about most common problems / causes and solutions with qmail mail here

First thing I did as usual is to send a test e-mail from Gmail to my Mailboxes on the mail server, the test mail was not received and in Gmail a failure to delivery notice was returned, I paste the TXT content of it as taken from Gmail's webmail -> Show Original menu:
 

                                                                                                                                                                                                                                                               
Delivered-To: hipodilski@gmail.com
Received: by 10.112.27.135 with SMTP id t7csp91961lbg;
        Mon, 29 Oct 2012 11:08:28 -0700 (PDT)
Received-SPF: pass (google.com: domain of  designates 10.60.171.72 as permitted sender) client-ip=10.60.171.72
Authentication-Results: mr.google.com; spf=pass (google.com: domain of  designates 10.60.171.72 as permitted sender) smtp.mail=; dkim=pass header.i=
Received: from mr.google.com ([10.60.171.72])
        by 10.60.171.72 with SMTP id as8mr17839903oec.140.1351534106946 (num_hops = 1);
        Mon, 29 Oct 2012 11:08:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:from:to:x-failed-recipients:subject:message-id:date
         :content-type:content-transfer-encoding;
        bh=GOkrGPurYWG9obiJDBWq6v3JHXdHlUebhVco7rIE73E=;
        b=mQemNDUu1Wl7d/VoIseXgXFbL0SdwMIY4MZH9GOm8TuRSVaU8oz80wdWt93zJTt/DR
         TEYTT6VRxUaDRAE2igBRLqjiSXdpZAJuBhoNA+bOTPwU53v+eaAUaV/7uHVHG0SYvF6u
         rkpc1Rbf41VYIDLthm+e7X8vFdaxqiYFiiHcih2stsAzgI9jAQy62SLlBytYRZeDc3po
         BXsb4SLm3+4kF4PuBlmhnCL+ba0hR3vQE5yC8/et0lPaxdSaJk0bHFkrjtmvg00fkyXo
         Pv+0dPJHvAInzHlPGtL+XHuvjZCq5XD5ZJjsajyAlG6J64z9dmziz8YM+qqA0KpNaF8+
         CVrA==
Received: by 10.60.171.72 with SMTP id as8mr17839903oec.140.1351534106944;
        Mon, 29 Oct 2012 11:08:26 -0700 (PDT)
MIME-Version: 1.0
Return-Path: <>
Received: by 10.60.171.72 with SMTP id as8mr20755231oec.140; Mon, 29 Oct 2012
 11:08:26 -0700 (PDT)
From: Mail Delivery Subsystem <mailer-daemon@googlemail.com>
To: hipodilski@gmail.com
X-Failed-Recipients: hipo@pc-freak.net
Subject: Delivery Status Notification (Failure)
Message-ID: <bcaec54edff658a23d04cd368e01@google.com>
Date: Mon, 29 Oct 2012 18:08:26 +0000
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Delivery to the following recipient failed permanently:

     hipo@pc-freak.net

Technical details of permanent failure:=20
Google tried to deliver your message, but it was rejected by the recipient =
domain. We recommend contacting the other email provider for further inform=
ation about the cause of this error. The error that the other server return=
ed was: 451 451 qq temporary problem (#4.3.0) (state 17).

----- Original message -----

DKIM-Signature: v=3D1; a=3Drsa-sha256; c=3Drelaxed/relaxed;
        d=3Dgmail.com; s=3D20120113;
        h=3Dmime-version:date:message-id:subject:from:to:content-type;
        bh=3DoaO9B2OZ1YJ19nwzUGkqXFVmVnakcfMdp7uW1TTA/u4=3D;
        b=3DQGaaKOgrnXxSa7X0ZjdmbG2/CWDPK10czq4n0YxHLRfX8N+pzJLHWXBFWmVWUNt=
yte
         rs8VrYu0BkdAE18MXS3x61cklvi/gk/eCUTzTm+L8fRu/Iiy6pZCr8S3Y6BWBN+5F1=
dm
         1LkL0mTpSHqVIoMB/fZwHIzz6q5tTqYSSNHX+hapu30eI5liyK5rbf2/4T9BhJ1VM0=
v+
         6NwupzAJK12jniKD8q9b4qJEhEoEqKKZrLKbTYiflHkAVMsg/C3v5zzwH+KZqsHP4W=
Us
         Tl/sHUcErXWOry1OrQXLNYR2K9vgqVdBUS5aoU2Jy1FgxbL/t5+XzB3tUK2mv43ttX=
0k
         wWbw=3D=3D
MIME-Version: 1.0
Received: by 10.60.171.72 with SMTP id as8mr10945566oec.140.1351262651282;
 Fri, 26 Oct 2012 07:44:11 -0700 (PDT)
Received: by 10.182.41.232 with HTTP; Fri, 26 Oct 2012 07:44:11 -0700 (PDT)
Date: Fri, 26 Oct 2012 16:44:11 +0200
Message-ID: <CAPk3ZemH=3D6U0oCihDqDwb9DT8EbE8mTjZou6zzw2LUHHO4ObbA@mail.gma=
il.com>
Subject:=20
From: Georgi Georgiev <hipodilski@gmail.com>
To: hipo@pc-freak.net
Content-Type: multipart/alternative; boundary=3Dbcaec54edff653f66804ccf75ab=
9

http://www.nybooks.com/blogs/nyrblog/2012/sep/04/jesus-vs-mao-interview-yua=
n-zhiming/

--=20
Georgi Georgiev
Mobile: +31644943358

After evaluating on qmail logs and various qmail components and basic qmail configurations, noticed the spamassassin spamd process is not running on the host. I've figured it out from qscanner-scanner.pl  in /var/log/qscan/qmail-queue.log, there were records saying, qmail-scanner can't pass incoming scanned mail to spand and thus failing

I onwards check in proclist to make sure qmail-queue.log suggestion is right, i.e.:

qmail:~# ps axu|grep -i spamd|grep -v grep
qmail:~#

As you see from my paste qmail scanner logs were right,spamd process died due to some memory leak bug or whatever. To temporary solve the problem I first launched spamd, via its init script:

 

qmail:~# /etc/init.d/spamassassn start

....

 

However it was clear, that in future spamd might unexpectedly terminate and this might ruin whole email receive processing again.

I've encountered on few qmail servers issues like this, so I knew of 3 work-arounds.

 

  • One is to use a tiny script set to run via cron job with superuser which checks every few minutes if spamd is running and if not re-launch it via the init script.

In some qmail installations, I've solved issues by using a tiny shell script – here you can download the script restart_spamd_if_crashed.sh

To use it download it to any directory lets say in /usr/local/bin and set cron job like:

qmail:~# crontab -u root -e

*/5 * * * * /usr/local/bin/restart_spamd_if_crashed.sh  >/dev/null 2>&1

 

  • Third and in my view best spamd crashes work-around  is to configure spamd to be constantly monitored and respawned whether found missing via daemontools.

To do so download those  spamd_daemontools_supervise_script.tar.gz archive and place it in /var/qmail/supervise (or wherever qmail/supervise dir is located) and create directory for spamd daemontools monitoring logs

qmail:~# cd /var/supervise
qmail:/var/supservice:# wget -q http://www.pc-freak.net/files/spamd_daemontools_supervise_script.tar.gz
qmail:/var/supervise:# tar -zxvvf spamd_daemontools_supervise_script.tar.gz
....
qmail:/var/supervise:# chmod +t spamd/
qmail:/var/supervise:# mkdir /var/log/spamd
qmail:/var/supervise:# chown -R qmaill:root /var/log/spamd
qmail:/var/supervise:# touch /var/log/spamd/current qmail:/var/supervise:# chown qmaill:nofiles /var/log/spamd/current

It is also generally good idea to check the content of scripts spamd/run and spamd/log/run, a common problem with those scripts is spamassassin might be custom installed in /usr/local/bin/spamd and not in the usual /usr/bin/spamd – spamd location is defined in spamd/run; as well as location of daemontools logging tool multilog might be not /usr/bin/multilog but in /usr/local/bin/multilog – depending on what kind of Qmail guide was used on qmail install time.

Finally, to make daemontools schedule for monitoring spamd service link it in /service dir:

qmail:~# ln -sf /var/supervise/qmail/ /service/qmail
qmail:~# ls -al /service/spamd lrwxrwxrwx 1 root root 27 Nov 8 14:38 spamd -> /var/qmail/supervise/spamd//

To check whether daemontools, started and watch over spamd check what is in /var/log/spamd/current and check the status of daemontools:

qmail:~# tail -n 5
qmail:~# ps ax|grep -i readproc|grep -v grep 27916 ? S 0:00 readproctitle service errors: .............................
qmail:~# tail -n 5 /var/log/spamd/current |tai64nlocal 2012-11-12

Whether, you're sure daemontools, now handles spamd, startup it is also recommended, you stop the on boot time /etc/init.d/spamassassin start-up.

qmail:~# mv /etc/rc2.d/S18spamassassin /etc/rc2.d/K81spamassassin

Of course if spamd is crashing due to some newly included anti-spam rule, which prevents spamassassin from starting, suggested fixes and even daemontools will be unable to "respawn" spamd. In most cases however, implementing any of above "fixes" will assure you a peaceful sleep.  

How to exclude sorbs.net for a particular IP address in Qmail Mail server install / Fix to Thunderbird mail sent error (Exploitable Server See: http://www.sorbs.net/lookup.shtml?xx.xx.xx.xx) error

Tuesday, November 1st, 2011

In the office, some of my colleagues has started receiving error messages, while trying to send mail with Thunderbird and Outlook Express
The exact error they handed to me reads like this:

An error occured while sending mail. The mail server responded: Exploitable Server See:
http://www.sorbs.net/lookup?xx.xx.xx.xx. Please check the message recipient

Here is also a screenshot, I’ve been sent via Skype with the error poping up on a Thunderbird installed on Windows host.

Typing the url http://www.sorbs.net/lookup?xx.xx.xx.xx lead me to sorbs.net to a page saying that the IP address of the mail client which is trying to send mail is blacklisted . This is not strange at all condireng that many of the office computers are running Windows and periodically get infected with Viruses and Spyware which does sent a number of Unsolicated Mail (SPAM).

The sorbs.net record for the IP seems to be an old one, since at the present time the office network was reported to be clear from malicious SMTP traffic.

The error sorbs.net disallowing the mail clients to send from the office continued for already 3 days, so something had to be done.

We asked the ISP to change the blacklisted IP address of xx.xx.xx.xx , to another one but they said it will take some time and they can’t do it in a good timely matter, hence to make mail sending work again with POP3 and IMAP protocols from the blacklisted IPs I had to set in the Qmail install to not check the xx.xx.xx.xx IP against mail blacklisting databases.

On qmail install disabling an IP check in RBLSMTPD is done through editting /etc/tcp.smtp and following recreate of /etc/tcp.smtp.cdb – red by qmailctl script start.
The exact line I put in the end of /etc/tcp.smtp to disable the RBLSMTPD check is:

xx.xx.xx.xx:allow,RBLSMTPD="",RELAYCLIENT="",QS_SPAMASSASSIN="0"

Further on to recreate /etc/tcp.smtp.cdb and reload the new cdb db records:

qmail:~# qmailctl cdb
qmail:~# qmailctl restart
...

Onwards, the sorbs.net IP blacklist issue was solved and all office computers from xx.xx.xx.xx succeeded in sending mails via SMTP.

Fix to mail forwarding error “Received-SPF: none (domain.com: domain at maildomain does not designate permitted sender hosts)

Tuesday, October 18th, 2011

I’m Configuring a new Exim server to relay / forward mail via a remote Qmail SMTP server
Even though I configured properly the exim to forward via my relaying mail server with host mail.domain.com, still the mail forwarding from the Exim -> Qmail failed to work out with an error:

Fix to mail forwarding error "Received-SPF: none (domain.com: domain at maildomain does not designate permitted sender hosts)

I pondered for a while on what might be causing this “mysterous” error just to realize I forgot to add the IP address of my Exim mail server in the Qmail relay server

To solve the error I had to add in /etc/tcp.smtp on my Qmail server a record for my Exim server IP address xx.xx.xx.xx, like so:

debian-server:~# echo 'xx.xx.xx.xx:allow,RELAYCLIENT="",QS_SPAMASSASSIN="0"' >> /etc/tcp.smtp

The QS_SPAMASSASSIN=”0″ as you might have guessed instructs Qmail not to check the received mails originating from IP xx.xx.xx.xx with spamassassin.

Finally on the Qmail server to load up the new tcp.smtp settings I had to rebuild /etc/tcp.smtp.cdb and restart qmail :

– reload qmail cdb

linux-server:/var/qmail# qmailctl cdb
Reloaded /etc/tcp.smtp.
- restart qmail

linux-server:/var/qmail# qmailctl restart
Restarting qmail:
* Stopping qmail-smtpdssl.
* Stopping qmail-smtpd.
* Sending qmail-send SIGTERM and restarting.
* Restarting qmail-smtpd.
* Restarting qmail-smtpdssl.

This solved the issue and now mails are forwarded without problems via the Qmail SMTPD.

Filter messages in Qmail with unwanted words, get rid of the Viagra annoying spam with Qtrap

Sunday, September 4th, 2011

Drop qmail received mail containing banned / unwanted words to get rid of Viagra and Sex related spam

Recently the annoying Viagra spam has emerged again. Therefore I decided to clean up some of the mails received to one of the qmail servers to protect users emailbox from this viagra peril.

To do so I’ve remember about an old script which used to be part of qmailrocks.org qmail install, the script is called qtrap and is able to filter emails based on list of specific mail contained words.
Since qmailrocks.org is gone (down) for some time and its still available only on few mirrored locations which by the way are not too easy to find I decided to write a little post on how qtrap.sh could be integrated quick & easy with any Qmail + Vpopmail install out there.

Hereby I include the description for qtrap.sh given by the script author:

“qtrap.sh script is applied on a per domain basis and serves as a “bad word” scanner to catch any spam that Spamassassin may have missed. This filter serves as the last defense against SPAM before it arrived in your inbox. I like this filter because it helps to get rid of any SPAM that happens to make it by Spamassassin. Without any protection at all, my mailbox gets a shit ton of SPAM every day. Within the first 3 months I enacted the Qtrap filter, Qtrap logged over 9,000 deleted SPAM messages, none of which were legitimate e-mails. My keyboard’s delete key was very appreciated the extra rest.

Any emails that are scanned and contain a banned word will be automatically deleted and logged by the qtrap script. A whitelist feature now exists so that individual addresses or domains can be exempt from the qtrap scan.”

Now as one might have general idea on what the script does. Here is the step by step qtrap.sh integration;

1. Create necessery qtrap directory and logs and set proper permissions

If the vpopmail is installed in /home/vpopmail , issue the following commands.

debian:~# cd /home/vpopmail
debian:~# mkdir -p qtrap/logs
debian:/home/vpopmail/qtrap# cd qtrap
debian:/home/vpopmail/qtrap# wget http://www.pc-freak.net/files/qtrap.sh
...
debian:/home/vpopmail/qtrap# cd ~
debian:~# touch /home/vpopmail/qtrap/logs/qtrap.logdebian:~# chown -R vpopmail:vchkpw /home/vpopmail/qtrapdebian:~# chmod -R 755 /home/vpopmail/qtrap

On older qmail installations it could be vpopmail is installed in /var/vpopmail if that’s the case, link /var/vpopmail to /home/vpopmail and go back to step 1. To link:

debian:~# ln -sf /var/vpopmail/ /home/vpopmail

2. Edit qtrap.sh to whitelist email addresses and build a ban words list

a) Include the email addresses mail arriving from which would not be checked by qtrap.sh

Inside qtrap.sh in line 63, there is a shell function whitelist_check(), the function looks like so:

whitelist_check () {
case $WHITELIST in
address@somewhere.com|address@somewhereelse.com)
echo $SENDER found in whitelist on `date "+%D %H:%M:%S"` >> /home/vpopmail/qtrap/logs/qtrap.log
exit 0;;
*)
;;
esac
}

By default the script has just two sample mails which gets whitelisted this is the line reading:

address@somewhere.com|address@somewhereelse.com

The whitelisted emails should be separated with a pipe, thus to add two more sample emails to get whitelisted by script the line should be changed like:

address@somewhere.com|address@somewhereelse.com|hipod@mymailserver.com|hipo@gmail.com

In order to whitelist an entire domain let’s say yahoo.com add a line to the above code like:

address@somewhere.com|address@somewhereelse.com|hipod@mymailserver.com|hipo@gmail.com|*yahoo.com

b) Defining the bad words ban list, mails containing them should not be delivery by qmail

The function that does check for the ban word list inside the script is checkall();, below is a paste from the script function:

checkall () {
case $BANNED_WORDS in
porn|PORN|Sex|SEX)
printout $BANNED_WORDS
echo MESSAGE DROPPED from $SENDER because of $BANNED_WORDS on `date "+%D %H:%M:%S"` >> /home/vpopmail/qtrap/logs/qtrap.log
exit 99;;
*)
;;
esac
}

checkall(); is located on line 74 in qtrap.sh, the exact list of banned words which the script should look for is located on line 76, the default qtrap.sh filters only mails containing just 4 words, e.g.:

porn|PORN|Sex|SEX)

To add the Viagra and VIAGRA common spam words to the list, modify it and expand like so:

porn|PORN|Sex|SEX|viagra|Viagra)

The delimiter is again | , so proceed further and add any unwanted spam words that are not common for any legit mails.

3. Install qtrap.sh to process all emails delivered to vpopmail

If its necessery to install the dropping of mails based on word filtering only to a single vpopmail virtualdomain do it with cmd:

debian:~# cd /home/vpopmail/domains/yourdomain.com
debian:/home/vpopmail/domains/yourdomain.com# touch .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# echo '| /home/vpopmail/qtrap/qtrap.sh' >> .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# echo "| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox" >> .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# chown vpopmail:vchkpw .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# cp -rpf .qmail-default .qmail-default.bak; mv .qmail-default.new .qmail-default
If however qtrap.sh needs to get installed for all existing vpopmail virtualdomains on the qmail server, issue a one liner bash script:

debian:~# cd /home/vpopmail/domains
debian:/var/vpopmail/domains# for i in *; do cd $i; echo "| /home/vpopmail/qtrap/qtrap.sh" >> $i/.qmail-default.new;
echo "| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox" >> $i/.qmail-default.new;
chown vpopmail:vchkpw .qmail-default.new; mv .qmail-default .qmail-default.old; mv .qmail-default.new .qmail-default; cd ..; done

This for loop will add ‘| /home/vpopmail/qtrap/qtrap.sh’ to all .qmail-default for all vpopmail domains.

Afterwards the .qmail-default file should contain the following two lines:

| /home/vpopmail/qtrap/qtrap.sh
| /home/vpopmail/bin/vdelivermail '' delete

A very important thing here you should consider that adding some common words, as let’s say hello or mail etc. could easily drop almost all the emails the qmail hands in to vpopmail.

Caution!! Never ever implement common words in the list of words !!
Always make sure the banned words added to qtrap.sh are words that are never enter an everyday legit email.

Another thing to keep in mind is that qtrap.sh doesn’t make a copy of the received message ,though it can easily be modified to complete this task.
Any mail that matches the banned words list will be dropped and lost forever.

4. Check if qtrap.sh is working

To check, if qtrap is working send mail to some mailbox located on the qmail server containing inside subject or mail message body the unwanted word defined inside qtrap.sh.

The mail should not be received in the mailbox to which its sent, if qtrap is working moreover qtrap.sh should log it inside it’s log file:

debian:~# cat /home/vpopmail/qtrap/logs/qtrap.log
MESSAGE DROPPED from hipo@mytestmail.com because of viagra on 09/03/11 11:34:19
MESSAGE DROPPED from support@mymailserver.com because of Viagra on 09/03/11 11:39:29

If the qtrap.log contains records similar to the one above, and the mail matching the banned word is not delivered, qtrap.sh is properly configured. If any issues check in qmail logs, they should have a good pointer on what went wrong with qtrap.sh invokation.

Note that I’ve integrated qtrap.sh to custom qmail install running on Debian Lenny 5.0 GNU/Linux.
If I have time I’ll soon test if its working fine on the latest stable Debian Squeeze and will report here in comments.
If however someone is willing to test if the script works on Debian Squeeze 6.0 or have tested it already please drop a comment to report if it works fine.
qtrap.sh, is a bit oldish and is not written to work too optimal therefore on some heavy loaded mail servers it can create some extra load and a bit delay the mail delivery. Thus when implementang one needs to consider the downsides of putting it in.

Also I was thinking tt might be nice if the script is rewritten to read the ban words and whitelist mails from files instead of as it is now as the words are hard coded in the script.
If I have some free time, I’ll probably do this, though I’m not sure if this is a too good idea as this might have a negative performance impact on the script execution time, as each instance of the script invoked should do one more operation of reading a file storing the ban words.

Well that’s pretty much it, enjoy 😉