Posts Tagged ‘mail’

How to force logrorate process logs / Make logrotate changes take effect immediately

Sunday, April 10th, 2016


Dealing with logrorate as admins we need to change or add new log-rorate configurations (on most Linux distributions configs are living uder

logrotate uses crontab to work. It's scheduled work, not as daemon, so usually no need to reload its configuration.
When the crontab executes logrotate, it will use your new config file automatically.

Most of the logrotate setups I've seen on various distros runs out of the /etc/cron.daily

$ ls -l /etc/cron.daily/logrotate 
-rwxr-xr-x 1 root root 180 May 18  2014 /etc/cron.daily/logrotate

Here is content of cron job scheduled script:

$ cat /etc/cron.daily/logrorate

#!/bin/sh /usr/sbin/logrotate /etc/logrotate.conf EXITVALUE=$? if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]" fi exit 0

Configurations change to lograte configs takes effect on next crontab run,
but what If you need to test your config you can also execute logrotate
on your own with below ommand:


logrotate -vf /etc/logrotate.conf 

If you encounter some issues with just modified or newly added logrorate script to check out the status of last logrorate executing bunch of log-rotate scripts run on Debian  / Ubuntu etc. deb based Linux:

cat /var/lib/logrotate/status

Or on RHEL, Fedora, CentOS Linux

cat /var/lib/logrotate.status

logrotate state -- version 2


"/var/log/syslog" 2016-4-9
"/var/log/dpkg.log" 2016-4-1
"/var/log/unattended-upgrades/unattended-upgrades.log" 2012-9-20
"/var/log/unattended-upgrades/unattended-upgrades-shutdown.log" 2013-5-17
"/var/log/apache2/" 2012-9-19
"/var/log/snort/portscan.log" 2012-9-12
"/var/log/apt/term.log" 2016-4-1
"/var/log/squid/access.log" 2015-3-21
"/var/log/mysql/mysql-slow.log" 2016-4-9
"/var/log/debug" 2016-4-3
"/var/log/mysql.log" 2016-4-9
"/var/log/squid/store.log" 2015-3-21
"/var/log/apache2/" 2012-9-19
"/var/log/daemon.log" 2016-4-3
"/var/log/munin/munin-update.log" 2016-4-9
"/var/log/unattended-upgrades/unattended-upgrades*.log" 2013-5-16
"/var/log/razor-agent.log" 2015-2-19
"/var/log/btmp" 2016-4-1
"/var/log/squid/*.log" 2014-11-24
"/var/log/munin/munin-graph.log" 2016-4-9
"/var/log/mysql/mysql.log" 2012-9-12
"/var/log/munin/munin-html.log" 2016-4-9
"/var/log/clamav/freshclam.log" 2016-4-3
"/var/log/munin/munin-node.log" 2016-1-23
"/var/log/" 2016-4-3
"/var/log/apache2/other_vhosts_access.log" 2016-4-3
"/var/log/exim4/rejectlog" 2012-9-12
"/var/log/squid/cache.log" 2015-3-21
"/var/log/messages" 2016-4-3
"/var/log/stunnel4/stunnel.log" 2012-9-19
"/var/log/apache2/php_error.log" 2012-10-21
"/var/log/ConsoleKit/history" 2016-4-1
"/var/log/rsnapshot.log" 2013-4-15
"/var/log/iptraf/*.log" 2012-9-12
"/var/log/snort/alert" 2012-10-17
"/var/log/privoxy/logfile" 2016-4-3
"/var/log/auth.log" 2016-4-3
"/var/log/postgresql/postgresql-8.4-main.log" 2012-10-21
"/var/log/apt/history.log" 2016-4-1
"/var/log/pm-powersave.log" 2012-11-1
"/var/log/proftpd/proftpd.log" 2016-4-3
"/var/log/proftpd/xferlog" 2016-4-1
"/var/log/zabbix-agent/zabbix_agentd.log" 2016-3-25
"/var/log/alternatives.log" 2016-4-7
"/var/log/mail.log" 2016-4-3
"/var/log/kern.log" 2016-4-3
"/var/log/privoxy/errorfile" 2013-5-28
"/var/log/aptitude" 2015-5-6
"/var/log/apache2/access.log" 2016-4-3
"/var/log/wtmp" 2016-4-1
"/var/log/pm-suspend.log" 2012-9-20
"/var/log/snort/portscan2.log" 2012-9-12
"/var/log/mail.warn" 2016-4-3
"/var/log/bacula/log" 2013-5-1
"/var/log/lpr.log" 2012-12-12
"/var/log/mail.err" 2016-4-3
"/var/log/tor/log" 2016-4-9
"/var/log/fail2ban.log" 2016-4-3
"/var/log/exim4/paniclog" 2012-9-12
"/var/log/tinyproxy/tinyproxy.log" 2015-3-25
"/var/log/munin/munin-limits.log" 2016-4-9
"/var/log/proftpd/controls.log" 2012-9-19
"/var/log/proftpd/xferreport" 2012-9-19
"/var/spool/qscan/qmail-queue.log" 2013-5-15
"/var/log/user.log" 2016-4-3
"/var/log/apache2/error.log" 2016-4-3
"/var/log/exim4/mainlog" 2012-10-16
"/var/log/privoxy/jarfile" 2013-5-28
"/var/log/cron.log" 2016-4-3
"/var/log/clamav/clamav.log" 2016-4-3


The timestamp date next to each of the rotated service log is when the respective log was last rorated

It is also a handy thing to rorate only a certain service log, lets say clamav-server, mysql-server, apache2 and nginx

logrorate /etc/logrorate.d/clamav-server
logrorate /etc/logrorate.d/mysql-server
logrotate /etc/logrotate.d/nginx

Share this on

Removing exim and installing qmail / Generate and install pseudo mta dummy package on Debian / Ubuntu etc. .deb based Linux

Thursday, March 10th, 2016

If you happen to be installing Qmail Mail server on a Debian or Ubuntu (.deb) based Linux, you will notice by default there will be some kind of MTA (Mail Transport Agent) already installed mail-transfer-agent package will be installed and because of Debian .deb package depedency to have an MTA always installed on the system you will be unable to remove Exim MTA without installing some other MTA (Postix / Qmail) etc.

This will be a problem for those like me who prefer to compile and install Qmail from source, thus to get around this it is necessery to create a dummy package that will trick the deb packaging depencies that actually mta-local MTA package is present on the server.

The way to go here is to use equivs (Circumvent debian package dependencies):

debian:~# apt-cache show equivs|grep -i desc -A 10

Description: Circumvent Debian package dependencies
 This package provides a tool to create trivial Debian packages.
 Typically these packages contain only dependency information, but they
 can also include normal installed files like other packages do.
 One use for this is to create a metapackage: a package whose sole
 purpose is to declare dependencies and conflicts on other packages so
 that these will be automatically installed, upgraded, or removed.
 Another use is to circumvent dependency checking: by letting dpkg
 think a particular package name and version is installed when it

Btw creating a .deb dummy package will be necessery in many other cases when you have to install from some third party debian repositories or some old and alrady unmaintaned deb-src packages for the sake of making some archaic software to resurrect somewhere, so sooner or later even if you're not into Mail servers you will certainly need equivs.

Then install equivs and go on proceeding creating the dummy mail-transport-agent package

debian:~# cd /tmp debian:~# cp -rpf /usr/share/doc/equivs/examples/mail-transport-agent.ctl . debian:~# equivs-build mail-transport-agent.ctl

Above command will build and package /tmp/mta-local_1.0_all.deb dummy package.
So continue and install it with dpkg as you use to install debian packages


debian:~# dpkg -i /tmp/mta-local_1.0_all.deb

From then on you can continue your standard LWQ – Life with Qmail or any other source based qmail installation with:




So that's it now .deb packaging system consistency will be complete so standard security package updates with apt-get and aptitude updates or dpkg -i third party custom software insatlls will not be breaking up any more.

Hope that helped someone 🙂





Share this on

Archive Outlook mail in Outlook 2010 to free space in your mailbox

Thursday, May 15th, 2014

If you're working in a middle or big sized IT company or corporation like IBM or HP, you're already sucked into the Outlook "mail whirlwind of corporate world" and daily flooded with tons of corporate spam emails with fuzzy business random terms like taken from Corporate Bullshit Generator

Many corporations, because probably of historic reasons still provide employees with small sized mailboxes half a gigabyte, a gigabyte or even in those with bigger user Mailboxes like in Hewlett Packard, this is usually no more than 2 Gigabytes.

This creates a lot of issues in the long term because usually mail communication in Inbox, Sent Items, Drafts Conversation History, Junk Email and Outbox grows up quickly and for a year or a year and a half, available Mail space fills up and you stop receiving email communication from customers. This is usually not too big problem if your Mailbox gets filled when you're in the Office (in office hours). However it is quite unpleasent and makes very bad impression to customers when you're in a few weeks Summar Holiday with no access to your mailbox and your Mailbox free space  depletes, then you don't get any mail from the customer and all the time the customer starts receiving emails disrupting your personal or company image with bouncing messages saying the "INBOX" is full.

To prevent this worst case scenario it is always a good idea to archive old mail communication (Items) to free up space in Outlook 2010 mailbox.
Old Outlook Archived mail is (Saved) exported in .PST outlook data file format. Later exported Mail Content and Contacts could be easily (attached) from those .pst file to Outlook Express, leaving you possibility to still have access to your old archived mail keeping the content on your hard drive instead on the Outlook Exchange Mailserver (freeing up space from your Inbox).

Here is how to archive your Outlook mail Calendar and contacts:


1. Click on the "File" tab on the top horizontal bar.Select "Cleanup Tools" from the options.

2. Click "Cleanup Tools" from the options.

3. Click on the "Archive this folder and all subfolders" option.

4. Select what to archive (e.g. Inbox, Drafts, Sent Items, Calendar whatever …)

5. Choose archive items older than (this is quite self-explanatory)

6. Select the location of your archive file (make sure you palce the .PST file into directory you will not forget later)

That's all now you have old mails freed up from Outlook Exchange server. Now make sure you create regular backups ot old-archived-mail.pst file you just created, it is a very good idea to upload this folder to encrypted file system on USB stick or use something like TrueCrypt to encrypt the file and store it to external hard drive, if you already don't have a complete backup corporate solution backuping up all your Laptop content.

Later Attaching or detaching exported .PST file in Outlook is done from:

File -> Open -> Open Outlook Data File


Once .PST file is opened and attached in Left Inbox pane you will have the Archived old mail folder appear.


You can change Archived name (like I did to some meaningful name) like I've change it to Archives-2013 by right clicking on it (Data File properties -> Advanced)

Share this on

Finding spam sending php scripts on multiple sites servers – Tracing and stopping spammer PHP scripts

Monday, April 14th, 2014

Spam has become a severe issue for administrators, not only for mail server admins but also for webshosting adms. Even the most secure spam protected mail server can get affected by spam due to fact it is configured to relay mail from other servers acting as web hosting sites.

Webhosting companies almost always suffer seriously from spam issues and often their mail servers gets blocked (enter spam blacklists), because of their irresponsible clients uploading lets say old vulnerable Joomla, WordPress without Akismet or proper spam handling plugin,a CMS which is not frequently supported / updated or custom client insecure php code.

What I mean is Shared server A is often configured to sent mail via (mail) server B. And often some of the many websites / scripts hosted on server A gets hacked and a spam form is uploaded and tons of spam start being shipped via mail server B.

Of course on mail server level it is possible to configure delay between mail sent and adopt a couple of policies to reduce spam, but the spam protection issue can't be completely solved thus admin of such server is forced to periodically keep an eye on what mail is sent from hosting server to mail server.

If you happen to be one of those Linux (Unix) webhosting admins who find few thousand of spammer emails into mail server logs or your eMail server queue and you can't seem to find what is causing it, cause there are multiple websites shared hosting using mainly PHP + SQL and you can't identify what php script is spamming by reviewing  Apache log / PHP files. What you can do is get use of:

PHP mail.log directive

Precious tool in tracking spam issues is a PHP Mail.log parameter, mail log paramater is available since PHP version >= 5.3.0 and above.
PHP Mail.log parameter records all calls to the PHP mail() function including exact PHP headers, line numbers and path to script initiating mail sent.

Here is how it is used:

1. Create empty PHP Mail.log file

touch /var/log/phpmail.log

File has to be writtable to same user with which Apache is running in case of Apache with SuPHP running file has to be writtable by all users.

On Debian, Ubunut Linux:

chown www:data:www-data /var/log/phpmail.log

On CentOS, RHEL, SuSE phpmail.log has to be owned by httpd:

chown httpd:httpd /var/log/phpmail.log

On some other distros it might be chown nobody:nobody etc. depending on the user with which Apache server is running.


2. Add to php.ini configuration following lines

mail.add_x_header = On
mail.log = /var/log/phpmail.log

PHP directive instructs PHP to log complete outbund Mail header sent by mail() function, containing the UID of the web server or PHP process and the name of the script that sent the email;

(X-PHP-Originating-Script: 33:mailer.php)

i.e. it will make php start logging to phpmail.log stuff like:


mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: — Headers: From: "Priority Mail" <status_93@pomoriemon> X-Mailer: MailMagic2.0 Reply-To: "Priority Mail" <> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="——
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: — Headers: From: "One Day Shipping" <status_44@pomoriemonastery.
org> X-Mailer: CSMTPConnectionv1.3 Reply-To: "One Day Shipping" <> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="—
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: — Headers: From: "Logistics Services" <customer.> X-Mailer: TheBat!(v3.99.27)UNREG Reply-To: "Logistics Services" <> Mime-Version: 1.0 Content-Type: mult
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: — Headers: From: "Priority Mail" <status_73@pomoriemonaster> X-Mailer: FastMailer/Webmail(versionSM/1.2.6) Reply-To: "Priority Mail" <> Mime-Version: 1.0 Content-Type: multipart/alternativ


On Debian / Ubuntu Linux to enable this logging, exec:

echo 'mail.add_x_header = On' >> /etc/php5/apache2/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/apache2/php.ini

I find it useful to symlink /etc/php5/apache2/php.ini to /etc/php.ini its much easier to remember php location plus it is a standard location for many RPM based distros.

ln -sf /etc/php5/apache2/php.ini /etc/php.ini

Or another "Debian recommended way" to enable mail.add_x_header logging on Debian is via:

echo 'mail.add_x_header = On' >> /etc/php5/conf.d/mail.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/conf.d/mail.ini

On Redhats (RHEL, CentOS, SuSE) Linux issue:

echo 'mail.add_x_header = On' >> /etc/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php.ini

3. Restart Apache

On Debian / Ubuntu based linuces:

/etc/init.d/apache2 restart

P.S. Normally to restart Apache without interrupting client connections graceful option can be used, i.e. instead of restarting do:

/etc/init.d/apache2 graceful

On RPM baed CentOS, Fedora etc.:

/sbin/service httpd restart


apachectl graceful

4. Reading the log

To review in real time exact PHP scripts sending tons of spam tail it:

tail -f /var/log/phpmail.log


mail() on [/var/www/remote-admin/wp-includes/class-phpmailer.php:489]: To: — Headers: Date: Mon, 14 Apr 2014 03:27:23 +0000 Return-Path: From: WordPress Message-ID: X-Priority: 3 X-Mailer: PHPMailer ( [version 2.0.4] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: — Headers: From: "Manager Elijah Castillo" <> X-Mailer: Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv: Gecko/20100111 Thunderbird/3.0.1 Reply-To: "Manager Elijah Castillo" <> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76017CC"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: — Headers: From: "Manager Justin Murphy" <> X-Mailer: Opera Mail/10.62 (Win32) Reply-To: "Manager Justin Murphy" <> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7603ED6"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: — Headers: From: "Manager Elijah Castillo" <> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv: Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Elijah Castillo" <> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7606308"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: — Headers: From: "Manager Justin Murphy" <> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv: Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Justin Murphy" <> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76086D1"


As you can see there is a junky spam mails sent via some spammer script uploaded under name .3a1a1c.php, so to stop the dirty bastard, deleted the script:

rm -f /var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php

It is generally useful to also check (search) for all hidden .php files inside directoring storing multiple virtualhost websites, as often a weirdly named hidden .php is sure indicator of either a PHP Shell script kiddie tool or a spammer form.

Here is how to Find all Hidden Perl / PHP scripts inside /var/www:

find . -iname '.*.php'

find . -iname '.*.pl*'


Reviewing complete list of all hidden files is also often useful to determine shitty cracker stuff

 find . -iname ".*"

Debugging via  /var/log/phpmail.log enablement is useful but is more recommended on development and staging (QA) environments. Having it enable on productive server with high amounts of mail sent via PHP scripts or just on dedicated shared site server could cause both performance issues, hard disk could quickly get and most importantly could be a severe security hole as information from PHP scripts could be potentially exposed to external parties.

Share this on

Outlook Express hotkeys (short keys) list – Use your MS Outlook efficiently

Thursday, March 13th, 2014

I'm foced to use Microsoft Outlook Express daily for my daily work in Hewlett Packard. Thus I decided to learn the hotkeys to make my Mail use more efficient. Here are list of Outlook Express hotkeys, hopefully to be useful to others too:

Print the selected message –  Ctrl+P
Send and receive mail –  Ctrl+M
Delete a mail message - DEL or Ctrl+D
Open or post a new message –  Ctrl+N
Open the Address Book - Ctrl+SHIFT+B
Reply to the message author –  Ctrl+R
Reply to all –  Ctrl+Shitf+R, Ctrl+G (news only)
Refresh news message and headers –  F5
Forward a message - Ctrl+F
Find Text - F3
Find a message - CTRL+Shift+F
Close Message - ESC
Check Names - Ctrl+K, Alt+S
Go to your Inbox - Ctrl+I
Go to the next message in the list - Ctrl+> or Ctrl+SHIFT+>
Go to the previous message in the list - Ctrl+< or Ctrl+SHIFT+<
View properties of a selected message - ALT+ENTER
View the full source of the message - Ctrl+F3
Go to the next unread mail message - Ctrl+U
Go to the next unread news conversation - Ctrl+Shift+U
Go to a folder –  Ctrl+Y
Open a selected message – Ctrl+O or ENTER
Mark a message as read - Ctrl+ENTER or Ctrl+Q
Move between the Folders list (if on), message list, preview pane, and Contacts list (if on) - TAB
Mark all news messages as read - Ctrl+SHIFT+A
Check spelling – F7
Insert signature - Ctrl+SHIFT+S
Send (post) a message –  Ctrl+ENTER or ALT+S
Select all messages –  Ctrl+A

Enjoy 🙂


Share this on

Quitting my job as IT Manager and moving to Further Horizons in Hewlett Packard

Friday, September 13th, 2013

International University College Logo IUC

I haven't blogged for a while for a plenty of reasons, I'm going through a change period in my life and as any change it is not easy.
This post will be not informative and will not teach any of my dear readers, anything on Computers its pretty personal but still for my friends it might cause interest.
Here is my personal life story over the last few months …

For a while I worked in a International University College situated in my hometown Dobrich. I was hired on position of IT Manager, and actually was doing a bit of E-Marketing to try to boost traffic to College's website – and mostly helping the old school hacker ad college system administrator over the last 10 yrs – Ertan to fix a bunch of Linux Mail / SQL and Webservers and some Windows machines. In college I learned from Ertan how to install and backups of restaurants software called BARBEQUE as well as how to fix problems with billing terminals situated in College Restaurant (3rd floor of building). Other of my work time I had to  fix infested Windows computers with viruses re-install Windowses and fix various printing and network problems of College's teachers, accountants, cash desk, marketing and rest of college  employees.

Talking about Ertan I should express my sincere tremendous Thanks (Thanks Ertan) to it for recommending me for this job position. Right before I started work in the college I was jobless for a while starting to get desperate that its impossible find work. Current IUC sysadmin – Mr. Ertan Geldiev is a remarkable man and one of the people that made great impression in my mind. Something I found interesting I can learn from Ertan was to get from his cheerful "admin" attitude. As a true hacker Ertan had this hacker attitude of playfulness I myself has for a while lost over the years. So seeing someone like this near my life make me a good favor and had a positive influence on myself.

I have learned a lot from Ertan during the 3 months and 3 days in International University College. Just for a bit of historic information earlier IUC was known as International College – Albena, also among Dobrich citizens known as The Dutch College – as earlier IUC had good relations with Dutch Universities and was issuing double degrees both Bulgarian and Dutch. Nowadays I'm not sure if still Double-degrees partnership between IUC and Dutch Universities exist, what I know for sure is college is issuing European Double Degrees in partnership with Cardiff Metropolitan University. I myself have earlier studied in the college and already know the place well thus will use this post to say a few words on my impressions on it …

International University College - one of top prestigious colleges in Eastern Europe

The college is a great place to be as you have chance to meet plenty of people both lecturers (professors), participate in the various events organized by College's as well as get involved in the many European Projects which are being handled by a European Projects department special department situated on the back of the College Building. Other positive about College is it is small and located on a peaceful town of Dobrich. This gives the bright people a lot of space for personal development, anyways on the other hand it can make you also a bad as Dobrich as a small city is a bit boring. The studies in College are good for students who want personal freedom as there is not too strict requirements for professors on how to teach.

Though college had help me grow, especially in my knowledge in Windows 7 and 8 (Ertan had a really good Windows background), I couldn't have the chance to develop myself too further in the long term. So my job offer to work in HP as Web and Middleware Implementation Engineer opened much broader opportunities for my long term IT career. Other reason I quit the College IT job was simply because I needed more money I had the vision to make a family with a girl from Belarus – Svetlana and in order to take care for her I need to earn good money. My official salary in the college was the funny for the position – 640 lv (though after a few months I was promised to have a raise and earn 400 EUR :)) . Such low sallary was for the reason I had the idea to continue studying in College and complete my Bachelor Business degree and we had agreed with the College CEO Mr. Todor Radev to extract part of my salary monthly and with that to pay my 1 semester tuition fee (2200 EUR) – necessary for my graduation assignment. Though completing the Bachelor is important phase to close in my life for a long time, I found for the moment more valuable to work for HP and earn normal living salary with which to possibly finance myself and create family with woman of my life (hopefully) in the short term.

In this post I want express my sincere thanks to all people in International College (Elena Urchenko, Krasimir – for helping me in my job duties), Pavel and Silvia for being a colleague for a while I worked partly in the Marketing Department.

Talking about Marketing Department what I did there is some Twitter Marketing (building some twitter followers) and wrote a tiny document with recommendation on how to optimize College website – (future version) – for better SEO ranking. This included complete analysis from user outlook to Indexing bots and site current code. 

Mr Docent Phd Todor Radev

I have to do a big underline on how great person the College President and UNI Rector – Docent Todor Radev is. I have already bitter experience studying for a while in a government universities when younger and I know from experience usually Rectors and Universities management of state universities is pure "Hell". Thanks to Mr. Todor Radev for he did me a big favor letting me quit  job just a week later (instead of 1 month as it is officially set by Bulgarian Dismissal Law and explicitly stated in my Work Contract. Also as a person my experience from Docent Radev is wonderful too. He is extremely intelligent, brilliant gentleman and  most importantly open minded and always open for innovation.

As a close up I would like to say Big Thanks to everyone which I worked with or met in International University College! Thanks guys for all your support and help, thanks for being work mates and friends for the time.

Share this on

10 must know and extremely useful Linux commands that every sys admin should know

Tuesday, July 30th, 2013

10 must know extremely useful gnu linux command line tools tips and tricks
There are plenty of precious command line stuff every admin should be aware on Linux. In this article I just decided to place some I use often and are interesting to know. Below commands are nothing special and probably many of experienced sys admins already know them. However I'm pretty sure novice admins and start-up Linux enthusiasts will find it useful. I know there much more to be said on the topic. So anyone is mostly welcome to share his used cmds.
1. Delete all files in directory except files with certain file extension

It is good trick to delete all files in directory except certain file formats, to do so:

root@linux:~# rm !(*.c|*.py|*.txt|*.mp3)

2. Write command output to multiple files (tee)

The normal way to write to file is by using redirect (to overwrite file) ">" or (to append to file) ">>";. However when you need to write output to multiple files there is a command called tee, i.e.:

root@linux:~# ps axuwwf | tee file1 file2 file3

3. Search for text in plain text file printing number of lines after match

Whether you need to print all number of lines after match of "search_text" use:

root@linux:~# grep -A 5 -i "search_text" text_file.txt

4. Show all files where text string is matched with GREP (Search for text recursively)

Searching for text match is extremely helpful for system administration. I use  grep recursive (capability) almost on daily basis:

root@websrv:/etc/dovecot# grep -rli text *

-l (instructs to only print file names matching string), -r (stands for recursive search), and -i flag (instructs grep to print all matches  inogoring case-sensitivity ( look for text nomatter if with capital or small letters)

5. Finding files and running command on each file type matched

In Linux with find command it is possible to search for files and run command on each file matched.
Lets say you we want to look in current directory for all files .swp (temporary) files produced so often by VIM and wipe them out:

root@linux:~# find . -iname '*.swp*' -exec rm -f {} \;

6. Convert DOS end of file (EOF) to UNIX with sed

If it happens you not have dos2unix command installed on Linux shell and you need to translate DOS end of file (\r\n – return carriage, new line) to UNIX's (\r – return carriage)), do it with sed:

root@linux:~# sed 's/.$//' filename

7. Remove file duplicate lines with awk:

cat test.txt
test duplicate
The brown fox jump over ...
Richard Stallman rox

root@linux:~# awk '!($0 in array) { array[$0]; print }' test.txt
test duplicate
The brown fox jump over ...
Richard Stallman rox

To remove duplicate text from all files in directory same can be easily scripped with bash for loop:

root@linux:~# for i in *; do
awk '!($0 in array) { array[$0]; print }' $i;

8. Print only selected columns from text file

To print text only in 1st and 7th column in plain text file with awk:

root@linux:~# awk '{print $1,$6;}' filename.txt ...

To print only all existing users on Linux with their respective set shell type:

root@linux:~# cat /etc/passwd|sed -e 's#:# #g'|awk '{print $1,$6;}'

9. Open file with VIM text editor starting from line

I use only vim for console text processing, and I often had to edit and fix file which fail to compile on certain line number. Thus use vim to open file for writing from necessary line num. To open file and set cursor to line 35 root@linux:~# vim +35 /home/hipo/current.c

10. Run last command with "!!" bash shorcut

Lets say last command you run is uname -a:

root@websrv:/home/student# uname -a
Linux websrv 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux

To re-run it simply type "!!":

root@websrv:/home/student# !!
uname -a
Linux websrv 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux



Share this on

FreeBSD Happy Birthday – FreeBSD OS turns 20 years old

Monday, June 24th, 2013

Happy Birrthday Freebsd / FreeBSD becomes 20 years old - Classic logo bsd avatar beastie mascot

On 19th of June FreeBSD had its 20 birthday! Its great joy for us the FreeBSD geeks 🙂
This is a bit outdated but as I'm a long years FreeBSD user I found myself obliged to mention the good news.

FreeBSD project was officially introduced for first time under the new name by David Greenman, Jordan Hubbard and Rod Grimes as fork of BSD (Berkley Software Distrbution 4.3).

Even more and less known is that big chunks of FreeBSD (kernel and userland) code are running on the shiny Mac OS X.

Below is mail which made up the name FreeBSD 🙂

To: (Interim 0.1.5)
Subject: Re: “386BSD” trademark (fwd)
From: David Greenman <>
Date: Sat, 19 Jun 93 17:26:02 -0700

> Okay folks.. taking new name suggestions.. we have:
> BSDFree86 – Rod, who is going with Jordans improved NON BSDI name..
> Free86BSD – Jordan, Rod likes this one two…
> – (F86BSD for short)
> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
>                v                                                                     v
>                v          This is the hat to drop yours in!       v
>                v                                                                     v
>                vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

How about just simply “FreeBSD”? No confusion, no fuss, seems like a good compromise to me. 🙂


FreeBSD Berkley Software Distrubution OS new logo happy birthday FreeBSD

Now 20 years after, FreeBSD itself had numerous forks very much in tradition of Free Software – most notable ones are of course BSD for desktop DragonFly BSD and PC-BSD.
As part of FreeBSD evolution nowadays FreeBSD kernel is ported for Debian Linux, i.e.:

debian:~# apt-cache show kfreebsd-source-8.1|grep -i description -A 2

Description: source code for kernel of FreeBSD 8.1 with Debian patches

 This package provides the source code for kernel of FreeBSD 8.1, base of

 a GNU/kFreeBSD system.

Description: source code for kernel of FreeBSD 8.1 with Debian patches

 This package provides the source code for kernel of FreeBSD 8.1, base of

 a GNU/kFreeBSD system.


 and can be run onas substitute for Linux kernel on most GNU / Linux distributions 

FreeBSD has unquestionably played key role over the last 20 yrs for development of Hacker Culture of Freedom and Free Software.

 Congrats and cheers to all FreeBSD users! 🙂  

Share this on

Linux: Fixing Qmail server qmail-smtpd port 25 slow (lagged) connect problem

Thursday, May 16th, 2013

qmail logo fixing qmail mail SMTP port 25 connect delays

After updating my Debian Squeeze to latest stable packages from repository with standard:
# apt-get update && apt-get upgrade

I routinely checked, if afterwards all is fine with Qmail?, just to find out connect to port 25 was hell delayed about 40-50 seconds before qmail responds with standard assigned Mail Greeting.
I Googled long time to see if I can find a post or forum thread discussing, exact issue, but though I found similar discussions I didn't found anything that exactly match problem. Thus I decided to follow the good old experimental try / fail method to figure out what causes it.

elow is pastes from telnet, illustrating delays in Qmail SMTP greeting respond:

# telnet localhost 25
Connected to localhost.
Escape character is '^]'.

# telnet localhost 25
Connected to localhost.
Escape character is '^]'.

# telnet localhost 25
Connected to localhost.
Escape character is '^]'.

I spend about 2 hours, checking Qmail for the standard so common errors, usually causing it to not work properly following my previous article testing qmail installation problems

After going, through all of possible causes the only clue for problems, were some slowness with spamassassin. This brought me the idea that something is done wrong with spamassassin .I tried disabling, Spamassassin Razon and Pyzor restarting spamd through (in my case done not via the standard start/stop debian script) but through daemontools with svc and qmailctl i.e.:

# svc -d /service/spamd
# svc -u /service/spamd
# svc -a /service/spamd

qmailctl restart
* Stopping qmail-smtpdssl.
* Stopping qmail-smtpd.
* Sending qmail-send SIGTERM and restarting.
* Restarting qmail-smtpd.
* Restarting qmail-smtpdssl.
* Restarting qmail-pop3d.
This doesn't help, so I continued trying to figure out, what is wrong .One assumption for slow  qmail-smtpd responce was of course slow DNS resolve issues. I checked /etc/resolv.conf to find out server is configured to use local  configured DJBDNS server as first line DNS resolver. I used djbdns for it is simple and easy to configure, however it is a bit obsolete so it was possible bottleneck. After commenting line to use localhost
and settings as primary DNS Google Public DNS, problem persisted so problems with hosts resolving was obviously not the problem.

I pondered for about 30 minutes, checking again all logs and checking machine processes. Just to remember before I experienced similar issues caused by unresolving RBL (blacklist IP) hosts. I checked configured SPF records in
(process list) and noticed following 4 hosts;

# ps auxwwf

7190 ?        S      0:00 tcpserver -vR -l /var/qmail/control/me -c 30 -u 89 -g 89 -x /etc/tcp.smtp.cdb 0 25 rblsmtpd -t0 -r -r -r -r qmail-smtpd /var/qmail/control/me /home/vpopmail/bin/vchkpw /bin/true

I checked one by one hosts and find out 1st two hosts in line are no longer resolving (blacklist is no longer accessible) as before:,

DNSBL (DNS blocklist) is configured on this host via /service/qmail-smtpd/run, hence to remove two unresolvable hosts forcing the weird qmail-smtpd connect delay I had to modify in it:




After a close examinations in mail server config /var/qmail/control/spfrules, found one other Unresolvable SPF Blacklist host configured ;
# cat /var/qmail/control/spfrules

To move that one I null-ed file:

# cat /dev/null > /var/qmail/control/spfrules

Finally to take affect all changes, launched Qmail start:

# qmailctl restart
Restarting qmail:
* Stopping qmail-smtpdssl.
* Stopping qmail-smtpd.
* Sending qmail-send SIGTERM and restarting.
* Restarting qmail-smtpd.
* Restarting qmail-smtpdssl.
* Restarting qmail-pop3d.

To check all was fine afterwards, again used telnet:

# telnet localhost 25
Connected to localhost.
Escape character is '^]'.
220 This is Mail Pc-Freak.NET ESMTP

Mail greeting now appears in about 2-3 seconds time.



Share this on

Linux: Understanding uptime command Load Avarage statistics / When load avarage is high?

Wednesday, May 8th, 2013

GNU / Linux load avarage explained load avarage from top command

There is probably no Linux system administrator who, don't have idea about system  Load Avarage. Most of admins however does have some brought idea about what kind of load avarage is critical but doesn't have good understanding on the 3 digits returned as a load avarage i.e. – load average: 2.47, 2.27, 2.02 shown in above ascii graphs ( generated by tload command).

What is Load Avarage ?

  •  The number of blocking processes in the run queue averaged over a certain time period.

A blocking process is a process that is waiting for something to continue. Typically, a process is waiting to use:

  •  CPU Time,  Disk Input / Output oper. or Network I / O

Thus logically the higher the Load Avarage, the more processes has to wait for access to CPU, HDD and Network I/O.

The most two common commands used where load avarage appear are;

w – who



mx:/home/hipo# w
 11:07:56 up 513 days,  1:04,  1 user,  load average: 1.92, 1.95, 1.84
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
hipo     pts/0     Thu19    0.00s  0.06s  0.02s sshd: hipo [priv]

mail:/home/hipo# uptime
 11:03:59 up 513 days,  1:00,  2 users,  load average: 2.11, 1.91, 1.81

Other common place to check load avarage is in top cmd:

mail:/home/hipo# top

Linux top command load avarage showing server system load 3 digits of load avarage explained

a) Optimum machine use – Load Avarage 1 

So what does load avarage: 1.74, 1.90, 1.83 really means? The 3 digits are showing system load avarage over the last 1, 5 and 15 minutes time. Meaning;


– before 1 minute system had a load of 1.74
– 5 minutes before it was 1.90
– and 15 minutes back 1.83

Usually Load Avarage of more than 1 is considered critical. If a system is working with a load avarage of 1 this means the system is working capacity. In best cases in terms of optimizing processes on server with hardware it is good the system is working in load of 0.70 or 0.80. Whether a traffic the machine gets is planned in most cases a load avarage of exactly 1 means machine hardware is properly utilized. However whether the load avarage is hitting over 1 this usually means you have to think about moving server to new hardware. It is general rule of thumb that if system load is exceeeding 0.70 it is time to migrate to better hardware.

b) Load avarage on Multi-core / Multiprocessor servers

Load avarage of 4 on 4 CPU cores server hardware is optimum one. Each core / CPU on machine should get maximum of load avarage 1. Load avarage of 1 means CPU is utilized in 100%. Load avarage of 4 on 4 CPU server hardware means all 4 processors are working in their maximum power of 100%. For people who have multi processor server the best way to show utilization is by running htop. There all 4 CPUs will show idle of 0%.

Hence rule to calculate normal load avarage for server is;

1 Load Avarage per CPU. Therefore for 24 CPU Intel Xeon hardware. Load Avarage under 1*24.00 = 24.00 is considered normal. On such a server whether load avarage jumps to 50.00 / 70.00 or above server becomes totally irresponsive and it is very likely to hang because of over-heating. Even if it continues working it will work extremely slow and even simply operations like ssh to it will become hardly possible and sometimes even access via ssh will be not possible.

Therefore Rule of Thumb for calculating which load avarage is okay for a server is;

Number of CPU / Cores should not exceed digit returned in Load Avarage stats

c) Critical – Load avarage >5 – A sure sign for unresponsive or soon to hang server

On Computers with just 1 CPU, load avarage of 5 is sure sign running services will lag brutally and server will become inaccessible. For multicore / multiprocessor servers big troubles can be expected, whether load avarage is about  1/2 of the maximum number of of Load Avarage; (for 8 CPU Multicore hardware). A load avarage of 8 + ( 1/2 * 8 ) = 12 is sure sign system is stoned and running services inaccessible.

d) load avarage: 1.74, 1.90, 1.83 – Is 1 / 5 or 15 minutes LA numbers more important to consider?

All are important however 5 and 15 minutes load avg. give better indication on what's happening with machine as current load can peak for just a second to a higher number, being misleading.

To get number of CPU / Cores use cmd;

mail:/home/hipo# grep 'model name' /proc/cpuinfo | wc -l

For more precise info on CPU type and model use;

mail:/home/hipo# cat /proc/cpuinfo

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 44
model name      : Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
stepping        : 2
cpu MHz         : 2400.094
cache size      : 12288 KB
physical id     : 0
siblings        : 12
core id         : 0
cpu cores       : 6
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips        : 4800.18
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

If you hit abnormal high load avarage, it is useful to check in top process list what is top process / processes causing highest system load. It is useful to run ps with following arguments

mail:/home/hipo# ps axuwwf

Look in STAT column. Processes in STAT have 3 states;

  • R – Running
  • S – Sleeping
  • D – Waiting for something

Usually processes with status of D – are ones causing problems. If you get D STAT-ed processes check further what's wrong with them and fix it. If there are none,  simply, number of clients using machine pop-ed up meaning you need to quickly move to better hardware host.

e) Getting notified via email whether load avarage exceeds certain value

A good way to get notified or do certain action like restarting Apache WebServer or other common process causing high loads is through monit. Monit is very usefukl for notifying on high load avarages or even better for Restarting processes imposing high loads.

You can always use a few liners shell script to mail to email or SMS2Email mailbox similar to this tiny shell script to restart apache on high load.There is also a Ruby lang tool – Scout to monitor and report high load avarages

Share this on