Installing qmailanalog and Generating daily qmail statistics with (qmail-stats.py) on Debian Lenny / Daily qmail statististics notification via email

Tuesday, 20th April 2010

Finding a decent software to generate daily qmail statistics is a really tough job this days. Before time I always used eitherqmailalizer or isoqlog.
Presently qmailalizer is completely abondoned piece of software and I cannot force it to work on 64 bit architecture.Isoqlog is another story, it’s supposed to work with qmailrocks, however my qmail installation is based on Bill’s Linux Qmail Toaster and for some strange reason it’s generating empty statistics. It could be that isoqlog is not generating statistics because the log files’s feed to be processed is not enough. Anyways still I cannot figure out the reason why I cannot make work Isoqlog with the Qmail Toaster.

I needed a way to however at least have an overview statistics of what is happening inside qmail. Of course qmailmrtg which is explained how to be installed in my previous post is providing with some overall information, though the information acquired through it is too general.
I’ve spend some enormous time searching for something that could inform me on various qmail statistics based on the qmail logs, before I could find and tweak the qmail-stats.py report script to become usable with qmailanalog

In the meantime It was necessery for me to investigate into qmailanalog and install it on the Debian system.
Initially I instlaled the qmailanalog from source, latest current source release can be obtained via D.J. Bernstein’s qmaialanlog download page
You won’t be able to compile the qmailanalog piece of code in debian until substitute in the source file: error.h the line:

extern int errno;

with#include <errno.h>

After the above change your source should succesfully compile.
Right after I compile it I realized there is a debian source package installer called:
qmailanalog-installer

So on Debian to install qmailanalog all I had to do was:

debian-server:~# apt-get install qmailanalog-installer
debian-server:~# /usr/bin/build-qmailanalog

Now as I already have qmailanalog properly installed on Debian I decided to test it with a script called qmail-logs.sh
You can download the qmail-logs.sh script from here

Here I quote what exactly is written in the qmail-logs.sh header in order to provide you with a general idea what the script does.

## Purpose:
## Wrapper for qmailanalog scripts. Will analyze qmail multilog
## files for deferrals, failures, overall statistics, or convert
## them to sendmail-style logs.

After executing the script I realized the script is not working properly because of some errors issued by scripts included within the qmailanalog package.
The faced problems and their solution with the qmailanalog: zsenders, zsuccesses, zfailures, zrecipients, zfailures and zrhosts scripts I’ve reported as a Debian bugs The qmailanalog bug report I’ve submitted can be seen here

The whole problem with qmailanalog scripts on Debian is that the “gnu more” binary arguments passed during the script operations are not correct and needs to be fixed.
Similar issues and their solution is explained in Qmailrocks Forums threads

After fixing the issues with qmailanalog I tried once again the qmail-logs.sh script, this time some of the functions provided by the script prooved working however some of them weren’t okay still.
Therefore I took some more time searching on the internet and I’ve found on a mailing list the qmail-stats.py script which worked like a charm with a minor modifications.

In order to have the qmail-stats.py working you need the tai64nfrac binary.You can download the latest current version of tai64nfrac on tai64nfrac’s website
Installation of tai64nfrac is pretty straight forward and comes to the following:

debian-server:~# wget http://archives.eyrie.org/software/system/tai64nfrac-1.4.tar.gz
debian-server:~# tar -zxvf tai64nfrac-1.4.tar.gz
debian-server:~# cd tai64nfrac-1.4
debian-server:~# make
debian-esrver:~# make install

After the install you should have the tai64nfrac in /usr/local/bin/tai64nfrac
Now let’s go back to the qmail-stats.py script. I’ve mirrored the qmail-stats.py script the qmail-stats.py script can be downloaded here

The script reports statistics on Qmail Logs following criterias:

Overall Email Server Statistics
Failure Statistics (Reasons for Failure)
Deferrals Statistics (Reasons for Deferrals)
Top Ten Senders Statistics
Top Ten Recipients Statistics

To make the script working all you have to edit in the script is the LOGFILE_PATH the rest is preconfigured by me already in the version of qmail-stats.py which is provided for download above.
If you choose to compile the qmailanalog from source you might also need to change the CMDS options, which includes the directory locations and commands from qmailanalog.

Now if you want to have the reports generated from qmail-stats.py, you have to setup the qmail-stats.py to be running via the cron daemon.

To do so open your root crontab and put in it:

# report daily qmail statistics05 01 * * * /usr/local/bin/qmail-stats.py | mail -s "Qmail Daily Statistics for $(date)" admin@domain.com

Now you should have the qmail-stats.py mailed to you every day at 01:05 early in the morning! 🙂

Share this on:

Download PDFDownload PDF

Tags:

10 Responses to “Installing qmailanalog and Generating daily qmail statistics with (qmail-stats.py) on Debian Lenny / Daily qmail statististics notification via email”

  1. Thibs says:
    Firefox 3.6.8 Firefox 3.6.8 Ubuntu 10.04 x64 Ubuntu 10.04 x64
    Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8

    If you’re agree, I’ll take some of your advce on http://qmailrocks.thibs.com/

    View CommentView Comment
  2. Thibs says:
    Firefox 5.0 Firefox 5.0 Windows XP Windows XP
    Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0

    Unfortunately, the qmailanalog-installer do not exists anymore under Debian Squeeze

    View CommentView Comment
    • admin says:
      Epiphany 2.30.6 Epiphany 2.30.6 Debian GNU/Linux x64 Debian GNU/Linux x64
      Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Version/5.0 Safari/531.2+ Debian/squeeze (2.30.6-1) Epiphany/2.30.6

      Yes too pity, even the backports repo doesn’t contain the package. Maybe the old package for Lenny would be working fine ?

      View CommentView Comment
  3. Monjinul Mobin says:
    Google Chrome 56.0.2924.87 Google Chrome 56.0.2924.87 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

    Hey There. I found your blog in google,your posts are extremely well written. Thanks for the post.

    I have followed above instruction but when i run qmail-stats.py then getting below errors. Could you please help me.

    File “/usr/local/bin/qmail-stats.py”, line 102, in
    os.chdir( LOGFILE_PATH )
    OSError: [Errno 13] Permission denied: ‘/var/log/qmail/qmail-send/’

    in details

    Failures
    —————-
    sort: cannot read: +2: No such file or directory
    awk: write failure (Broken pipe)
    awk: close failed on file /dev/stdout (Broken pipe)
    Reasons for failure

    One line per reason for delivery failure. Information on each line:
    * del is the number of deliveries that ended for this reason.
    * xdelay is the total xdelay on those deliveries.

    del xdelay reason

    ***************************************************************
    Deferrals
    —————-
    sort: cannot read: +2: No such file or directory
    awk: close failed on file /dev/stdout (Broken pipe)
    Reasons for deferral

    One line per reason for deferral. Information on each line:
    * del is the number of deliveries that ended for this reason.
    * xdelay is the total xdelay on those deliveries.

    del xdelay reason

    ***************************************************************
    Top Ten Senders
    —————-
    sort: cannot read: +7: No such file or directory
    awk: write failure (Broken pipe)
    awk: close failed on file /dev/stdout (Broken pipe)
    * xdelay is the total xdelay incurred by this sender.
    * tries is the number of delivery attempts (success, failure, deferral).
    Senders
    * sbytes is the number of bytes successfully received from this sender.
    * recips is the number of recipients (success plus failure).
    * rbytes is the number of bytes from this sender, weighted by recipient.
    One line per sender. Information on each line:
    * mess is the number of messages sent by this sender.
    mess bytes sbytes rbytes recips tries xdelay sender
    * bytes is the number of bytes sent by this sender.

    ***************************************************************
    Top Ten Recipients
    —————-
    sort: cannot read: +4: No such file or directory
    awk: write failure (Broken pipe)
    awk: close failed on file /dev/stdout (Broken pipe)
    * xdelay is the total xdelay incurred by this recipient.
    * tries is the number of delivery attempts (success, failure, deferral).
    * sbytes is the number of bytes successfully delivered to this recipient.
    sbytes mess tries xdelay recipient
    * mess is the number of messages sent to this recipient (success plus failure).
    One line per recipient. Information on each line:
    Recipients

    View CommentView Comment
    • admin says:
      Firefox 52.0 Firefox 52.0 Windows 7 x64 Edition Windows 7 x64 Edition
      Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0

      Hi Mobin,

      Is your /var/log/qmail/qmail-send/ directory existing and does it have proper permissions, could be your qmail log file directory is different?

      Regards,

      Georgi

      View CommentView Comment
      • Monjinul Mobin says:
        Google Chrome 56.0.2924.87 Google Chrome 56.0.2924.87 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
        Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

        Hi Georgi

        Yes /var/log/qmail/qmail-send/ directory exist and its permission is set as qmaill nofiles. qmail-stats.py also have same permission. Pleas help me.

        Thanks

        View CommentView Comment
  4. Monjinul Mobin says:
    Google Chrome 56.0.2924.87 Google Chrome 56.0.2924.87 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

    Hi Georgi,

    Sorry it is actually working partially. Please see the below stats. It is only showing summary but not showing Top ten sender,receiver,Failures or Deferrals. Please help

    Completed messages: 1397
    Recipients for completed messages: 1645
    Total delivery attempts for completed messages: 1646
    Average delivery attempts per completed message: 1.17824
    Bytes in completed messages: 407985081
    Bytes weighted by success: 461962077
    Average message qtime (s): 1.88326

    Total delivery attempts: 1654
    success: 1544
    failure: 102
    deferral: 8
    Total ddelay (s): 1724.399932
    Average ddelay per success (s): 1.116839
    Total xdelay (s): 2421.575779
    Average xdelay per delivery attempt (s): 1.464072
    Time span (days): 0.0756242
    Average concurrency: 0.370615

    ***************************************************************

    Failures
    —————-
    sort: cannot read: +2: No such file or directory
    awk: write failure (Broken pipe)
    awk: close failed on file /dev/stdout (Broken pipe)
    Reasons for failure

    One line per reason for delivery failure. Information on each line:
    * del is the number of deliveries that ended for this reason.
    * xdelay is the total xdelay on those deliveries.

    del xdelay reason

    ***************************************************************
    Deferrals
    —————-
    sort: cannot read: +2: No such file or directory
    awk: close failed on file /dev/stdout (Broken pipe)
    Reasons for deferral

    One line per reason for deferral. Information on each line:
    * del is the number of deliveries that ended for this reason.
    * xdelay is the total xdelay on those deliveries.

    del xdelay reason

    ***************************************************************
    Top Ten Senders
    —————-
    sort: cannot read: +7: No such file or directory
    awk: write failure (Broken pipe)
    awk: close failed on file /dev/stdout (Broken pipe)
    * xdelay is the total xdelay incurred by this sender.
    * tries is the number of delivery attempts (success, failure, deferral).
    Senders
    * sbytes is the number of bytes successfully received from this sender.
    * recips is the number of recipients (success plus failure).
    * rbytes is the number of bytes from this sender, weighted by recipient.
    One line per sender. Information on each line:
    * mess is the number of messages sent by this sender.
    mess bytes sbytes rbytes recips tries xdelay sender
    * bytes is the number of bytes sent by this sender.

    ***************************************************************
    Top Ten Recipients
    —————-
    sort: cannot read: +4: No such file or directory
    awk: write failure (Broken pipe)
    awk: close failed on file /dev/stdout (Broken pipe)
    * xdelay is the total xdelay incurred by this recipient.
    * tries is the number of delivery attempts (success, failure, deferral).
    * sbytes is the number of bytes successfully delivered to this recipient.
    sbytes mess tries xdelay recipient
    * mess is the number of messages sent to this recipient (success plus failure).
    One line per recipient. Information on each line:
    Recipients

    View CommentView Comment
    • admin says:
      Firefox 52.0 Firefox 52.0 Windows 7 x64 Edition Windows 7 x64 Edition
      Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0

      Hi Mobin,

      Try to run the script with

      strace qmainanalog

      and see closely the output could be some log file expected under a location is not there but on a different location, check the script concent where the /var/log/qmail paths as described in script are matching yours.

      But anyways strace should tell you a bit more, Hope this helps.

      Best
      Georgi

      View CommentView Comment
  5. Colin says:
    Google Chrome 102.0.0.0 Google Chrome 102.0.0.0 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36

    Hello

    Those failures with cannot read +4 and similar messages are caused by old deprecates parameters being used with the sort command used in most of the z* scripts in the qmailanalog package. Have a look here

    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578395

    at a Debian bug report for some more information. I updated my qmailanalog pscripts as indicated and it fixed it for me..

    Regards

     

    Colin

    View CommentView Comment

Leave a Reply

CommentLuv badge