Posts Tagged ‘Solve’

How to solve qmail-inject: fatal: qq temporary problem (#4.3.0) Qmail and Qmail Scanner problems on Debian Linux Wheezy

Monday, October 16th, 2017

fix-qq-error-qmail-inject-qq-temporary-problem-4.3.0-PoweredBy-qmail-Logo

Below QMAIL error
 

qmail-inject: fatal: qq temporary problem (#4.3.0)


occured to me right after upgraded from Debian Linux Squeeze 6 to Debian 7 Wheezy,

qmail-inject: fatal: qq temporary problem (#4.3.0) is really terrible error and I only experienced that error in my Thunderbird during sending mails, mail receiving doesn't work either, so as normally when there are problems with Qmail its a lot of puzzling until you get it.

There is no even trace in logs on what might be causing it, strangely enough nothing in qmail-smtpd, qmail-send logs, the mail server and all components seemed to work perfectly fine I checked whether there are libraries that are missing with a small loop line as follows:
 

 

root@pcfreak:/var/log/qmail/qmail-smtpd# for i in $(ls -1 /var/qmail/bin/*); do ldd $i |grep -i "not found"; done
root@pcfreak:/var/log/qmail/qmail-smtpd# 

 


The absence of result indicates, all binaries are properly linked and no found mmissing libraries.

After investigating closely what might be wrong and reading comments on Thibs QmailRocks Install Qmail-Scanner page, I realied
the error might be caused, because of problems with suid perl, as I already checked my earlier post in which I seemed to have faced the same qmail-inject: fatal: qq temporary problem (#4.3.0) error on Debian Wheezy and explained the possible reasons what might be causing the qq qmail error  here as well

 

and a related issue I experienced earlier with qmail scanner unable to create files in previous article here Suid Perl no longer available as a package and therefore because of the inability of perl to run as root anymore in Debian Wheezy, qmail-scanner-queue.pl script did not work either.
 

root@pcfreak:/downloads/simscan-1.4.0# 320  echo "hi, testing." > /tmp/mailtest.txt
root@pcfreak:/downloads/simscan-1.4.0# env QMAILQUEUE=/var/qmail/bin/qmail-scanner-queue /var/qmail/bin/qmail-inject you@yourdomain.com < /tmp/mailtest.txt
YOU HAVEN'T DISABLED SET-ID SCRIPTS IN THE KERNEL YET!
FIX YOUR KERNEL, PUT A C WRAPPER AROUND THIS SCRIPT, OR USE -u AND UNDUMP!
qmail-inject: fatal: qq permanent problem (#5.3.0)

root@pcfreak:/downloads/simscan-1.4.0# /var/qmail/bin/qmail-scanner-queue
YOU HAVEN'T DISABLED SET-ID SCRIPTS IN THE KERNEL YET!
FIX YOUR KERNEL, PUT A C WRAPPER AROUND THIS SCRIPT, OR USE -u AND UNDUMP!

 

 

A short note to make here is qmail-scanner-queue and qmail-scanner-queue.pl are set with suid bit set as follows:
 

root@pcfreak:/home/hipo/info# ls -al /var/qmail/bin/{qmail-scanner-queue,qmail-scanner-queue.pl}
-rwsr-sr-x 1 qscand qscand   6814 окт 14 17:22 /var/qmail/bin/qmail-scanner-queue*
-rwsr-sr-x 1 qscand qscand 158880 окт 14 23:52 /var/qmail/bin/qmail-scanner-queue.pl*

Good to say here is qmail-scanner-queue is a suid wrapper binary that actually invokes qmail-scanner-queue.pl
 

root@pcfreak:/downloads/simscan-1.4.0# su hipo
hipo@pcfreak:/downloads/simscan-1.4.0$ /var/qmail/bin/qmail-scanner-queue.pl -g
perlscanner: generate new DB file from /var/spool/qscan/quarantine-events.txt
hipo@pcfreak:/downloads/simscan-1.4.0$ exit

root@pcfreak:/downloads/simscan-1.4.0# cp /downloads/qmail-scanner-2.11st/contrib/logrotate.qmail-scanner /etc/logrotate.d/qmail-scanner
root@pcfreak:/downloads/simscan-1.4.0# chmod 644 /etc/logrotate.d/qmail-scanner
root@pcfreak:/downloads/simscan-1.4.0# cd /downloads/qmail-scanner-2.11st/contrib
root@pcfreak:/downloads/qmail-scanner-2.11st/contrib# chmod 755 test_installation.sh
root@pcfreak:/downloads/qmail-scanner-2.11st/contrib# ./test_installation.sh -doit
Sending standard test message – no viruses… 1/4
qmail-inject: fatal: qq temporary problem (#4.3.0)
Bad error. qmail-inject died


This are the other things, I've done to fix possible permission issues

root@pcfreak:/downloads/qmail-scanner-2.11st/contrib#  sudo -u qscand /var/qmail/bin/qmail-scanner-queue.pl -z
root@pcfreak:/downloads/qmail-scanner-2.11st/contrib# chown qscand:qscand /var/spool/qscan/qmail-scanner-queue-version.txt

In /etc/sudoers add following lines:

root@pcfreak:~# vim /etc/sudoers

ALL ALL=(qscand) NOPASSWD: /var/qmail/bin/qmail-scanner-queue.pl
##necroleak ALL=(ALL) ALL
Defaults        env_keep +="TCPREMOTEIP TCPREMOTEHOST TCPREMOTEINFO REMOTE_OS REMOTE_COUNTRY QS_SPAMASSASSIN SA_ONLYDELETE_HOST SA_WHITELIST
"


root@pcfreak:/downloads/qmail-scanner-2.11st/contrib# cat /etc/sudoers

# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#

Defaults    env_reset

# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root    ALL=(ALL) ALL
hipo    ALL=(ALL) ALL
ALL ALL=(qscand) NOPASSWD: /var/qmail/bin/qmail-scanner-queue.pl
##necroleak ALL=(ALL) ALL
Defaults        env_keep +="TCPREMOTEIP TCPREMOTEHOST TCPREMOTEINFO REMOTE_OS REMOTE_COUNTRY QS_SPAMASSASSIN SA_ONLYDELETE_HOST SA_WHITELIST"
# Allow members of group sudo to execute any command
# (Note that later entries override this, so you might need to move
# it further down)
%sudo ALL=(ALL) ALL
#
#includedir /etc/sudoers.d

In case you wonder why we put the line in /etc/sudoers:
 

Defaults        env_keep +="TCPREMOTEIP TCPREMOTEHOST TCPREMOTEINFO REMOTE_OS REMOTE_COUNTRY QS_SPAMASSASSIN SA_ONLYDELETE_HOST SA_WHITELIST"


The reason behind this is that by default sudo resets the environment variables when executing the command. Thus qmail-scanner cannot recognize the important info regarding the incoming mail and treats everything as coming from localhost, which leads to passing everything without scanning. The above line preserves the important ENV variables for qmail-scanner.

In /etc/sudoers add following lines:
 

root@pcfreak:/downloads/qmail-scanner-2.11st/contrib# vim /var/qmail/bin/qmail-scanner-queue.pl


Right after comments or in Line 1  ADD
 

my $real_uid="qscand";

my $effective_uid="qscand";


Also somewhere in the beginning of scripts lets say after above two variable declarations add:
 

$ENV{'PATH'}='/bin:/usr/bin';
$whoami = getpwuid($<) || "unknown";
if($whoami ne "qscand") {
    exec("/usr/bin/sudo -u qscand /var/qmail/bin/qmail-scanner-queue.pl") || die;
}


To prevent your users logged in on physical console and via SSH it is necessery to disable emergency logs for users in syslog / rsyslog, otherwise due to bug, users will logged in will get flooded with messages such as:
 

Message from syslogd@pcfreak at Oct 15 16:43:04 …
 qmail-scanner-queue.pl: qmail-scanner[6834]: Clear:RC:0(95.142.194.223):SA:0(-2.0/2.0): 2.959242 10574 admin@notify.vk.com hipo@www.pc-freak.net Светлана_Георгиева_оставила_Вам_личное_соо� <36b63ec9a0ce7ecc570de2fcbba6ed73@localhost.localdomain> 1508074981.6836-1.pcfreak:6219 1508074981.6836-0.pcfreak:545

 

open /etc/rsyslog.conf and find the line starting with:
 

root@pcfreak:~# vim /etc/rsyslog.conf

*.emerg

add
;user.none

right after it so it reads like:

*.emerg;user.none

How to solve crashing spamassassin spamd service preventing QMAIL mail server to properly deliver mails / Setting spamd work via daemontools

Monday, November 12th, 2012

How to solve crashing spamd, script to restart spamd on failure, set spamd to run via daemontools, configure spamd to be restarted from monit service

On my home router configured qmail install, I have recently noticed I receive e-mails only sent via contact forms on the few websites hosted there. I'm subscribed to Debian newsletter, as well as usually receive about 10 emails and few spam mails every day, so after a few days of reduced emails on my email (receiving only e-mail notification about  blog issued comments), I logically suspected something is not properly working with the qmail installation.
My first logical guesses was the usual Qmail problems I've previously experienced through the years, earlier I blogged about most common problems / causes and solutions with qmail mail here

First thing I did as usual is to send a test e-mail from Gmail to my Mailboxes on the mail server, the test mail was not received and in Gmail a failure to delivery notice was returned, I paste the TXT content of it as taken from Gmail's webmail -> Show Original menu:
 

                                                                                                                                                                                                                                                               
Delivered-To: hipodilski@gmail.com
Received: by 10.112.27.135 with SMTP id t7csp91961lbg;
Mon, 29 Oct 2012 11:08:28 -0700 (PDT)
Received-SPF: pass (google.com: domain of  designates 10.60.171.72 as permitted sender) client-ip=10.60.171.72
Authentication-Results: mr.google.com; spf=pass (google.com: domain of  designates 10.60.171.72 as permitted sender) smtp.mail=; dkim=pass header.i=
Received: from mr.google.com ([10.60.171.72])
by 10.60.171.72 with SMTP id as8mr17839903oec.140.1351534106946 (num_hops = 1);
Mon, 29 Oct 2012 11:08:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:from:to:x-failed-recipients:subject:message-id:date
:content-type:content-transfer-encoding;
bh=GOkrGPurYWG9obiJDBWq6v3JHXdHlUebhVco7rIE73E=;
b=mQemNDUu1Wl7d/VoIseXgXFbL0SdwMIY4MZH9GOm8TuRSVaU8oz80wdWt93zJTt/DR
TEYTT6VRxUaDRAE2igBRLqjiSXdpZAJuBhoNA+bOTPwU53v+eaAUaV/7uHVHG0SYvF6u
rkpc1Rbf41VYIDLthm+e7X8vFdaxqiYFiiHcih2stsAzgI9jAQy62SLlBytYRZeDc3po
BXsb4SLm3+4kF4PuBlmhnCL+ba0hR3vQE5yC8/et0lPaxdSaJk0bHFkrjtmvg00fkyXo
Pv+0dPJHvAInzHlPGtL+XHuvjZCq5XD5ZJjsajyAlG6J64z9dmziz8YM+qqA0KpNaF8+
CVrA==
Received: by 10.60.171.72 with SMTP id as8mr17839903oec.140.1351534106944;
Mon, 29 Oct 2012 11:08:26 -0700 (PDT)
MIME-Version: 1.0
Return-Path: <>
Received: by 10.60.171.72 with SMTP id as8mr20755231oec.140; Mon, 29 Oct 2012
11:08:26 -0700 (PDT)
From: Mail Delivery Subsystem <mailer-daemon@googlemail.com>
To: hipodilski@gmail.com
X-Failed-Recipients: hipo@www.pc-freak.net
Subject: Delivery Status Notification (Failure)
Message-ID: <bcaec54edff658a23d04cd368e01@google.com>
Date: Mon, 29 Oct 2012 18:08:26 +0000
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Delivery to the following recipient failed permanently:
hipo@www.pc-freak.net
Technical details of permanent failure:=20
Google tried to deliver your message, but it was rejected by the recipient =
domain. We recommend contacting the other email provider for further inform=
ation about the cause of this error. The error that the other server return=
ed was: 451 451 qq temporary problem (#4.3.0) (state 17).
----- Original message -----
DKIM-Signature: v=3D1; a=3Drsa-sha256; c=3Drelaxed/relaxed;
d=3Dgmail.com; s=3D20120113;
h=3Dmime-version:date:message-id:subject:from:to:content-type;
bh=3DoaO9B2OZ1YJ19nwzUGkqXFVmVnakcfMdp7uW1TTA/u4=3D;
b=3DQGaaKOgrnXxSa7X0ZjdmbG2/CWDPK10czq4n0YxHLRfX8N+pzJLHWXBFWmVWUNt=
yte
rs8VrYu0BkdAE18MXS3x61cklvi/gk/eCUTzTm+L8fRu/Iiy6pZCr8S3Y6BWBN+5F1=
dm
1LkL0mTpSHqVIoMB/fZwHIzz6q5tTqYSSNHX+hapu30eI5liyK5rbf2/4T9BhJ1VM0=
v+
6NwupzAJK12jniKD8q9b4qJEhEoEqKKZrLKbTYiflHkAVMsg/C3v5zzwH+KZqsHP4W=
Us
Tl/sHUcErXWOry1OrQXLNYR2K9vgqVdBUS5aoU2Jy1FgxbL/t5+XzB3tUK2mv43ttX=
0k
wWbw=3D=3D
MIME-Version: 1.0
Received: by 10.60.171.72 with SMTP id as8mr10945566oec.140.1351262651282;
Fri, 26 Oct 2012 07:44:11 -0700 (PDT)
Received: by 10.182.41.232 with HTTP; Fri, 26 Oct 2012 07:44:11 -0700 (PDT)
Date: Fri, 26 Oct 2012 16:44:11 +0200
Message-ID: <CAPk3ZemH=3D6U0oCihDqDwb9DT8EbE8mTjZou6zzw2LUHHO4ObbA@mail.gma=
il.com>
Subject:=20
From: Georgi Georgiev <hipodilski@gmail.com>
To: hipo@www.pc-freak.net
Content-Type: multipart/alternative; boundary=3Dbcaec54edff653f66804ccf75ab=
9
http://www.nybooks.com/blogs/nyrblog/2012/sep/04/jesus-vs-mao-interview-yua=
n-zhiming/
--=20
Georgi Georgiev
Mobile: +31644943358

After evaluating on qmail logs and various qmail components and basic qmail configurations, noticed the spamassassin spamd process is not running on the host. I've figured it out from qscanner-scanner.pl  in /var/log/qscan/qmail-queue.log, there were records saying, qmail-scanner can't pass incoming scanned mail to spand and thus failing

I onwards check in proclist to make sure qmail-queue.log suggestion is right, i.e.:

qmail:~# ps axu|grep -i spamd|grep -v grep
qmail:~#

As you see from my paste qmail scanner logs were right,spamd process died due to some memory leak bug or whatever. To temporary solve the problem I first launched spamd, via its init script:

 

qmail:~# /etc/init.d/spamassassn start

....

 

However it was clear, that in future spamd might unexpectedly terminate and this might ruin whole email receive processing again.

I've encountered on few qmail servers issues like this, so I knew of 3 work-arounds.

 

  • One is to use a tiny script set to run via cron job with superuser which checks every few minutes if spamd is running and if not re-launch it via the init script.

In some qmail installations, I've solved issues by using a tiny shell script – here you can download the script restart_spamd_if_crashed.sh

To use it download it to any directory lets say in /usr/local/bin and set cron job like:

qmail:~# crontab -u root -e

*/5 * * * * /usr/local/bin/restart_spamd_if_crashed.sh  >/dev/null 2>&1

 

  • Third and in my view best spamd crashes work-around  is to configure spamd to be constantly monitored and respawned whether found missing via daemontools.

To do so download those  spamd_daemontools_supervise_script.tar.gz archive and place it in /var/qmail/supervise (or wherever qmail/supervise dir is located) and create directory for spamd daemontools monitoring logs

qmail:~# cd /var/supervise
qmail:/var/supservice:# wget -q https://www.pc-freak.net/files/spamd_daemontools_supervise_script.tar.gz
qmail:/var/supervise:# tar -zxvvf spamd_daemontools_supervise_script.tar.gz
....
qmail:/var/supervise:# chmod +t spamd/
qmail:/var/supervise:# mkdir /var/log/spamd
qmail:/var/supervise:# chown -R qmaill:root /var/log/spamd
qmail:/var/supervise:# touch /var/log/spamd/current qmail:/var/supervise:# chown qmaill:nofiles /var/log/spamd/current

It is also generally good idea to check the content of scripts spamd/run and spamd/log/run, a common problem with those scripts is spamassassin might be custom installed in /usr/local/bin/spamd and not in the usual /usr/bin/spamd – spamd location is defined in spamd/run; as well as location of daemontools logging tool multilog might be not /usr/bin/multilog but in /usr/local/bin/multilog – depending on what kind of Qmail guide was used on qmail install time.

Finally, to make daemontools schedule for monitoring spamd service link it in /service dir:

qmail:~# ln -sf /var/supervise/qmail/ /service/qmail
qmail:~# ls -al /service/spamd lrwxrwxrwx 1 root root 27 Nov 8 14:38 spamd -> /var/qmail/supervise/spamd//

To check whether daemontools, started and watch over spamd check what is in /var/log/spamd/current and check the status of daemontools:

qmail:~# tail -n 5
qmail:~# ps ax|grep -i readproc|grep -v grep 27916 ? S 0:00 readproctitle service errors: .............................
qmail:~# tail -n 5 /var/log/spamd/current |tai64nlocal 2012-11-12

Whether, you're sure daemontools, now handles spamd, startup it is also recommended, you stop the on boot time /etc/init.d/spamassassin start-up.

qmail:~# mv /etc/rc2.d/S18spamassassin /etc/rc2.d/K81spamassassin

Of course if spamd is crashing due to some newly included anti-spam rule, which prevents spamassassin from starting, suggested fixes and even daemontools will be unable to "respawn" spamd. In most cases however, implementing any of above "fixes" will assure you a peaceful sleep.  

How to debug mod_rewrite .htaccess problems with RewriteLog / Solve mod_rewrite broken redirects

Friday, September 30th, 2011

Its common thing that CMS systems and many developers custom .htaccess cause issues where websites depending on mod_rewrite fails to work properly. Most common issues are broken redirects or mod_rewrite rules, which behave differently among the different mod_rewrite versions which comes with different versions of Apache.

Everytime there are such problems its necessery that mod_rewrite’s RewriteLog functionality is used.
Even though the RewriteLog mod_rewrite config variable is well described on httpd.apache.org , I decided to drop a little post here as I’m pretty sure many novice admins might not know about RewriteLog config var and might benefit of this small article.
Enabling mod_rewrite requests logging of requests to the webserver and process via mod_rewrite rules is being done either via the specific website .htaccess (located in the site’s root directory) or via httpd.conf, apache2.conf etc. depending on the Linux / BSD linux distribution Apache config file naming is used.

To enable RewriteLog near the end of the Apache configuration file its necessery to place the variables in apache conf:

1. Edit RewriteLog and place following variables:

RewriteLogLevel 9
RewriteLog /var/log/rewrite.log

RewriteLogLevel does define the level of logging that should get logged in /var/log/rewrite.log
The higher the RewriteLogLevel number defined the more debugging related to mod_rewrite requests processing gets logged.
RewriteLogLevel 9 is actually the highest loglevel that can be. Setting the RewriteLogLevel to 0 will instruct mod_rewrite to stop logging. In many cases a RewriteLogLevel of 3 is also enough to debug most of the redirect issues, however I prefer to see more, so almost always I use RewriteLogLevel of 9.

2. Create /var/log/rewrite.log and set writtable permissions

a. Create /var/log/rewrite.log

freebsd# touch /var/log/rewrite.log

b. Set writtable permissons

Either chown the file to the user with which the Apache server is running, or chmod it to permissions of 777.

On FreeBSD, chown permissions to allow webserver to write in file, should be:

freebsd# chown www:www /var/log/rewrite.log

On Debian and alike distros:

debian:~# chown www-data:www-data /var/log/rewrite.log

On CentOS, Fedora etc.:

[root@centos ~]# chown httpd:httpd /var/log/rewrite.log

On any other distribution, you don’t want to bother to check the uid:gid, the permissions can be set with chmod 777, e.g.:

linux# chmod 777 /var/log/rewrite.log

Next after RewriteLog is in conf to make configs active the usual webserver restart is required.

To restart Apache On FreeBSD:

freebsd# /usr/local/etc/rc.d/apache2 restart
...

To restart Apache on Debian and derivatives:

debian:~# /etc/init.d/apache2 restart
...

On Fedora and derivive distros:

[root@fedora ~]# /etc/init.d/httpd restart
...

Its common error to forget to set proper permissions to /var/log/rewrite.log this has puzzled me many times, when enabling RewriteLog’s logging.

Another important note is when debugging for mod_rewrite is enabled, one forgets to disable logging and after a while if the /var/log partition is placed on a small partition or is on an old server with less space often the RewriteLog fills in the disk quickly and might create website downtimes. Hence always make sure RewriteLog is disabled after work rewrite debugging is no longer needed.

The way I use to disable it is by commenting it in conf like so:

#RewriteLogLevel 9
#RewriteLog /var/log/rewrite.log

Finally to check, what the mod_rewrite processor is doing on the fly its handy to use the well known tail -f

linux# tail -f /var/log/rewrite.log

A bunch of time in watching the requests, should be enough to point to the exact problem causing broken redirects or general website malfunction.
Cheers 😉

Solve ALSA audio and mic issues on Lenovo Thinkpads on Debian and Ubuntu Linux

Wednesday, January 11th, 2012

Since I've blogged about my recent skype issues. I've played a lot with pulseaudio, alsa, alsa-oss to experimented a lot until I figured out why Skype was failing to properly delivery sound and record via my embedded laptop mic.

Anyways, while researching on the cause of my Thinkpad r61 mic issues, I've red a bunch of blog posts by people experiencing microphone oddities with Lenovo Thinkpads

Throughout the search I come across one very good article, which explained that in many cases the Thinkpad sound problems are caused by the snd-hda-intel alsa kernel module. snd-hda-intel fails to automatically set proper sb model type argument during Linux install when the soundcard is initialized with some argument like options snd-hda-intel model=auto

Hence, the suggested fix which should resolve this on many Thinkpad notebooks is up to passing the right module argument:

To fix its neceessery to edit /etc/modprobe.d/alsa-base.conf .

debian:~# vim /etc/modprobe.d/alsa-base.conf

Find the line in the file starting with:
options snd-hda-intel model=

and substitute with:

options snd-hda-intel model=thinkpad

Finally a restart of Advaned Linux Sound Architecture (alsa) is required:

debian:~# /etc/init.d/alsa restart
...

At most cases just restarting the alsa via its init script is not enough, since the ssnd-hda-intel kernel module is already in use by some program or something, so its best to do a reboot to make sure the module is loaded with the new model=thinkpad argument.

My exact laptop sound card model is:

debian:~# lspci |grep -i audio
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03)

After changing the module and using alsamixer and aumix to make sure mic is unmuted and its volume is high enough, mic sound rec works fine.